5 / 8 Data Leakage & Preprocessing

Time Traveler's Tell

A team predicts next-day stock price movement using a random forest. They shuffle all rows randomly and use k-fold cross-validation, achieving 85% accuracy in testing but the model performs near-randomly in live trading. What is the most likely cause?

A

K-fold cross-validation with 5 folds is too few for a dataset with high variance

B

Temporal leakage: random shuffling let future data end up in training folds used to predict past-dated validation rows, an ordering that can't happen in live deployment

C

The model has too many trees, causing it to memorize noise in the training set

D

Standardization was applied to the price data, distorting the true feature scale