Foundations 0/0

Foundations

Training deep networks

Convolutional, recurrent and residual networks, and the normalisation, regularisation and loss choices that let deep networks train and generalise. Residual connections and layer normalisation carry straight into the transformer.

0 / 14 solved

Questions

0/14 solved
Question Module Tags Difficulty Success
1. Team Norm Regularization & Loss Techniques
normalizationlayernorm +1
Medium —
2. Quit While You're Ahead Regularization & Loss Techniques
early-stoppingoverfitting +1
Medium —
3. What inductive bias does mixup introduce? Regularization & Loss Techniques
Machine LearningTraining & Optimization
Medium 40%
4. What exactly does focal loss down-weight? Regularization & Loss Techniques
Machine LearningTraining & Optimization
Medium —
5. How does Group Normalization differ from Batch Normalization? Regularization & Loss Techniques
Machine LearningDeep Learning Architecture
Medium —
6. What is inverted dropout? Regularization & Loss Techniques
Machine LearningDeep Learning Architecture
Medium 86%
7. Four Thirds Variance Regularization & Loss Techniques
batch-normalizationinference-mode +2
Hard 100%
8. What problem does focal loss address? Regularization & Loss Techniques
Machine LearningTraining & Optimization
Medium 100%
9. Decoupled Discipline Regularization & Loss Techniques
weight-decayl2-regularization +1
Medium —
10. How can label smoothing affect knowledge distillation? Regularization & Loss Techniques
Machine LearningTraining & Optimization
Medium —
11. Same Stencil, Every Pixel Architectures (CNN / RNN)
cnnconvolution +1
Easy 100%
12. The Additive One Architectures (CNN / RNN)
residual-connectionsvanishing-gradient +2
Hard 88%
13. Zoom Out With Every Layer Architectures (CNN / RNN)
cnnreceptive-field +1
Medium 60%
14. Forget Me Not Architectures (CNN / RNN)
rnnlstm +1
Medium —