What moments does Adam estimate?

Question: What does the Adam optimizer maintain for each parameter? A) Only the raw parameter value B) Only the final validation score C) Exponential moving averages of first and second moments of gradients D) The exact Hessian matrix for every batch Correct: C Explanation: Adam uses estimates of the first moment, similar to momentum, and the second moment, related to squared gradients. These are used to adapt learning rates for individual parameters. Topic: advanced ML / optimization / Adam