6 / 8 Generalization

Same Distribution, Different Day

A spam classifier is trained on emails collected in 2023, all drawn independently from the same underlying distribution as required by standard statistical learning theory. In 2026 it starts performing poorly because spammers have changed their writing style, so P(email content) has shifted even though P(spam | content) for old-style spam is unchanged. What best describes this situation?

A

This is expected i.i.d. behavior, since the model was trained and evaluated on data from the same year

B

This has nothing to do with the i.i.d. assumption, only with overfitting to the 2023 training set

C

A violation of the i.i.d. assumption via covariate shift: the input distribution changed between training and deployment while the true labeling relationship stayed fixed

D

A violation of the i.i.d. assumption via label shift: the proportion of spam vs. non-spam emails changed while feature distributions per class stayed fixed

E

This is a case of underfitting, since the model never fit the 2023 training data well enough