Training & Optimization

Losses, optimizers, regularization, and neural network training mechanics

0 / 12 solved

Practice Problems

0 / 12 completed

Downhill Tuning 73% 44/60 solved

Hidden Layer Workhorse 80% 44/55 solved

Memorized the Noise 32% 34/105 solved

Under where? 60% 33/55 solved

Folded Test Run 81% 35/43 solved

What problem does focal loss address? 100% 2/2 solved

What moments does Adam estimate? 100% 2/2 solved

How does AdamW differ from Adam with L2 regularization?

What objective does SAM approximately optimize?

What inductive bias does mixup introduce?

What exactly does focal loss down-weight?

How can label smoothing affect knowledge distillation?