Chain Reaction

Backprop computes the gradient of the scalar loss L with respect to a weight w in an early layer by multiplying local derivatives along the path from w to L. Which rule underlies this, and why is applying it in reverse (output-to-input) more efficient than forward-mode differentiation for a network with millions of weights but a single scalar loss?

The mean value theorem, approximating each gradient via finite differences between nearby weight values

The chain rule, applied in reverse from output to input, reusing shared intermediate gradients, cheap when inputs (many weights) vastly outnumber outputs (one loss)

The chain rule, applied forward from input to output, recomputing gradients independently for every weight from scratch

The product rule combined with L'Hopital's rule to bound each partial derivative