Where The Curve Turns Twice

You sweep the width multiplier kk of a ResNet on a CIFAR-10 training set carrying 15% label noise, holding the optimizer, epochs and data fixed, and record final errors:

width kk train error test error
4 24.0% 33.1%
10 7.9% 27.4%
18 0.0% 36.2%
26 0.0% 29.0%
64 0.0% 21.3%

Where does the interpolation threshold sit, and what does the classical bias-variance reading get wrong here?

A

At k=26k = 26, the first width past the test-error peak, since the threshold is read off the test curve

B

At k=10k = 10, the last width before test error starts climbing; past that point extra capacity only fits noise, so the 21.3% at k=64k = 64 has to be a seed artifact

C

At k=18k = 18, the smallest width that drives train error to zero; the classical reading stops at k=10k = 10 and predicts every larger model is worse, missing the second descent from 36.2% to 21.3%

D

At k=64k = 64, since the widest model interpolates most easily; the 36.2% spike at k=18k = 18 is run-to-run variance