Matched Pair

For multi-class classification, softmax output is almost always paired with cross-entropy loss rather than, say, mean squared error. What is the main practical reason for this pairing?

Cross-entropy is the only loss function that is differentiable

Softmax requires a loss with no computed derivative to avoid vanishing gradients

The gradient of the combined loss with respect to the pre-softmax logits simplifies to (predicted probabilities minus true labels), giving clean, well-scaled gradients

Softmax outputs are integers, and only cross-entropy accepts integer-valued inputs

MSE cannot be computed when there are more than two output classes