Goldilocks Weights

A deep sigmoid network initialized with weights drawn from a wide N(0,1) distribution shows gradients shrinking to near zero by the time backprop reaches the earliest layers. Which technique directly addresses this by scaling the initial weight variance according to each layer's fan-in and fan-out?

Raising the learning rate for the earliest layers only

Xavier/Glorot (or He) initialization, which sets weight variance based on fan-in/fan-out to keep activation and gradient magnitudes roughly stable across layers

L2 weight regularization, which shrinks all weights uniformly during training

Batch normalization applied only at the final output layer