Two Steps Downhill

Minimize $f(w) = w^2$ by gradient descent from $w_0 = 3$ .

w = 3.0
for _ in range(2):
    w = w - eta * grad(w)     # grad(w) is the derivative of w**2

What is w after the loop when eta = 0.1, and what happens instead when eta = 1.1?

$w = 2.40$ . After the first step the gradient is already zero, so the second iteration changes nothing.

$w = 1.92$ . With $\eta = 1.1$ every step multiplies $w$ by $-1.2$ , so the iterate flips sign and grows without bound.

$w = 2.43$ . With $\eta = 1.1$ it still converges, just in fewer steps.

$w = 4.32$ . The update adds the gradient, so both learning rates climb the loss.