ML Foundations

L1, L2, and L-infinity distance, metric axioms, cosine similarity, and KL divergence.

Mean, variance, covariance vs correlation, and estimating them from samples.

Dot products and angles, orthogonality, unit vectors, linear and quadratic forms, diagonal matrices.

k-NN, choosing k, cross-validation, confusion matrices, and the cost of it all.

Fit a Gaussian generative classifier, read a covariance matrix, and know when the decision boundary is linear.

The three regression losses and when to use each, plus the bias trick.

Derivatives, gradient descent and SGD, convexity, and positive semidefinite matrices.

Decision boundaries, probability thresholds, the cross-entropy objective, and why it is convex with no closed form.

The perceptron and its convergence, the soft-margin SVM, margins, support vectors, slack, and the role of C.

Multiclass linear models, decision regions, decision-tree growth and stopping, Gini vs misclassification, overfitting.

Overfitting vs underfitting, the statistical learning assumption, and covariate vs label shift.

Bagging vs boosting, random forests, and the first ideas of neural nets: activations, softmax, parameter counting.