Warm Start, Cool Finish

A transformer training run uses a linear learning-rate warmup for the first few thousand steps followed by a cosine decay schedule down to a small final value, and separately clips the global gradient norm to 1.0 at every step. Which statement correctly matches each technique to its purpose?

Warmup rescales gradients per layer to equalize their norms, cosine decay adjusts the momentum coefficient over time, and gradient clipping only applies to the embedding layer to prevent token collapse

Warmup avoids destabilizing large updates from an uninitialized optimizer state early in training, cosine decay smoothly reduces the step size as the model approaches convergence, and gradient clipping caps outlier gradient spikes (common with transformers) so a single bad batch cannot blow up the weights

Warmup exists to speed up the very first epoch by using an unusually large learning rate, cosine decay is used to periodically restart training from scratch, and gradient clipping increases the effective batch size

Warmup and gradient clipping both serve to prevent overfitting late in training, while cosine decay controls the weight decay coefficient rather than the learning rate