The Saturation Trap

It reduces the number of floating point operations needed to compute the attention scores

Without scaling, dot products of high-dimensional random vectors have variance that grows with $d$, producing large-magnitude logits that push softmax into saturated regions with near-zero gradients

It converts the dot product into a proper probability distribution so the softmax output sums to 1

It compensates for the fact that $Q$ and $K$ are computed with different weight matrices than $V$