No Shortcut Formula

You train a logistic regression spam classifier on bag-of-words features (raw word counts as a high-dimensional sparse input vector). Unlike ordinary linear regression's normal equation, why must you fit the weights with an iterative method like gradient descent instead of solving directly?

Sparse feature matrices cannot ever be inverted

The cross-entropy loss's gradient runs through the nonlinear sigmoid, so setting the gradient to zero yields no closed-form algebraic solution, even though the loss remains convex

Bag-of-words counts are categorical, not numeric, so they cannot enter a linear system

The cross-entropy loss is non-convex for high-dimensional sparse inputs