Will More Rows Help

Two models, each retrained from scratch at three dataset sizes, with error (lower is better):

training rows A train A validation B train B validation
1,000 0.28 0.31 0.02 0.24
10,000 0.29 0.30 0.05 0.17
100,000 0.30 0.30 0.08 0.13

You can buy one million more labeled rows. Which model is worth spending it on?

A

Model B, because its training error is climbing as rows are added, which is the signature of underfitting.

B

Model B: its two curves are still closing (gap 0.22, then 0.12, then 0.05), so more rows are still buying validation accuracy.

C

Model A: its validation error is worse, and more data always helps the weaker model the most.

D

Neither: both curves are flat past 10,000 rows, so dataset size no longer matters for either model.