Back
All Modules Machine Learning Deep Learning Regularization & Loss Techniques

Regularization & Loss Techniques

BatchNorm vs GroupNorm, dropout, weight decay, and loss-level tricks: label smoothing, focal loss, mixup.

0 / 10 solved
0/10 solved
Question Tags Difficulty Success
1. Team Norm
normalizationlayernorm +1
 Medium —
2. Quit While You're Ahead
early-stoppingoverfitting +1
 Medium —
3. What inductive bias does mixup introduce?
Machine LearningTraining & Optimization
Medium —
4. What exactly does focal loss down-weight?
Machine LearningTraining & Optimization
Medium —
5. How does Group Normalization differ from Batch Normalization?
Machine LearningDeep Learning Architecture
Medium —
6. What is inverted dropout?
Machine LearningDeep Learning Architecture
Medium 80%
7. Four Thirds Variance
batch-normalizationinference-mode +2
 Hard —
8. What problem does focal loss address?
Machine LearningTraining & Optimization
Medium —
9. Decoupled Discipline
weight-decayl2-regularization +1
 Medium —
10. How can label smoothing affect knowledge distillation?
Machine LearningTraining & Optimization
Medium —

Learning Resources

The Little Book of Deep Learning (free)

Up next

10 of 10 questions here are still unsolved.

Next module Architectures (CNN / RNN)