Activations & Optimization

Activation functions, adaptive optimizers, learning-rate schedules, momentum, and sharpness-aware minimization.

0 / 6 solved

Question Tags Difficulty Success

1. Dead End Street

activationsrelu

Medium —

2. The Slow Roll

optimizationmomentum

Medium —

3. First Step Is Alpha

adambias-correction +2

Medium 100%

4. Warm Start, Cool Finish

learning-rategradient-clipping

Medium —

5. Where The Decay Goes

adamwweight-decay +2

Hard —

6. What objective does SAM approximately optimize?

Machine LearningTraining & Optimization

Medium —

Learning Resources

The Little Book of Deep Learning (free)

Up next

6 of 6 questions here are still unsolved.

Next module Regularization & Loss Techniques