Three-Way Split Decision

During model development you tune hyperparameters (like learning rate and regularization strength) by repeatedly checking performance on one held-out set, then report a final number on a completely separate set only once, at the very end. Why keep these as two distinct sets instead of one?

Splitting into validation and test only matters when the dataset is small

Repeatedly using the same held-out set for tuning leaks information into your choices, so it stops being an unbiased estimate; a separate, untouched test set is needed for an honest final performance estimate

Validation sets are only needed for deep learning, not for classical ML models

The test set is used to compute the loss function during training, so it must be separate from validation