?
What does nested cross-validation estimate?
PostedJun 21, 2026
Question: A researcher performs hyperparameter selection using inner cross-validation and reports the average score from an independent outer cross-validation loop. What does the outer-loop score most directly estimate?
A) The expected performance of every hyperparameter configuration considered in the inner loop
B) The generalization error of the model refitted once on the complete dataset using the globally best hyperparameters
C) The generalization performance of the full model-selection procedure, including hyperparameter tuning
D) An unbiased estimate of the Bayes error whenever the outer folds are stratified
Correct: C
Explanation: The outer loop evaluates the entire model-selection process. Hyperparameters are selected only from the corresponding outer-training partition, and the untouched outer-validation partition evaluates the selected procedure. It does not estimate Bayes error and does not separately estimate every candidate configuration.
Topic: advanced ML / model selection / nested cross-validation