First Step Is Alpha

Adam with the paper's defaults: $\alpha = 0.001$, $\beta_1 = 0.9$, $\beta_2 = 0.999$, $\epsilon = 10^{-8}$, and $m_0 = v_0 = 0$.

At step $t = 1$ a particular parameter sees gradient $g_1 = 0.1$.

$m_1 = \beta_1 m_0 + (1-\beta_1) g_1, \qquad v_1 = \beta_2 v_0 + (1-\beta_2) g_1^2$

$\hat{m}_1 = \frac{m_1}{1-\beta_1^{,1}}, \qquad \hat{v}_1 = \frac{v_1}{1-\beta_2^{,1}}, \qquad \Delta\theta = -\alpha,\frac{\hat{m}_1}{\sqrt{\hat{v}_1}+\epsilon}$

How large is $|\Delta\theta|$?