Question 817

The Validation Loader That Keeps Moving

An image classifier reports a validation accuracy that swings by two to three points between consecutive epochs, and re-running evaluation on the same checkpoint gives a different number every time.

train_tf = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ColorJitter(0.4, 0.4, 0.4),
    transforms.ToTensor(),
])

train_loader = DataLoader(ImageFolder(train_dir, transform=train_tf), shuffle=True)
val_loader   = DataLoader(ImageFolder(val_dir,   transform=train_tf), shuffle=False)

What is wrong, and what is the fix?

A

This is test-time augmentation, which lowers variance, so the fix is only to average several augmented passes per validation image.

B

The validation set is re-randomised on every pass, so each evaluation scores a different perturbed dataset; give validation a deterministic transform such as Resize plus CenterCrop.

C

Nothing is wrong: every split must go through the same transform so training and validation see the same distribution, and the swing is normal epoch-to-epoch noise.

D

Training images are leaking into validation through the shared transform object, which is why the score moves; construct a separate Compose object for each loader.