Foundations
Training deep networks
Convolutional, recurrent and residual networks, and the normalisation, regularisation and loss choices that let deep networks train and generalise. Residual connections and layer normalisation carry straight into the transformer.
Questions
Question Module Difficulty Success
5. How does Group Normalization differ from Batch Normalization? Regularization & Loss Techniques Medium —
10. How can label smoothing affect knowledge distillation? Regularization & Loss Techniques Medium —