Which Calibration Score Becomes The Threshold

You have a fitted regressor and a held-out calibration set of n=500n = 500 points. For each you compute the nonconformity score si=∣yi−f^(xi)∣s_i = |y_i - \hat f(x_i)| and sort the 500 scores in ascending order. You want split conformal intervals C^(x)=f^(x)±q^\hat C(x) = \hat f(x) \pm \hat q at miscoverage α=0.10\alpha = 0.10.

Which sorted calibration score is q^\hat q?

A

The 50th smallest score, since α=0.10\alpha = 0.10 points at the lower 10% of the calibration scores

B

The 450th smallest score, the ordinary 90th percentile of the 500 calibration scores

C

The 451st smallest score, because q^\hat q is the ⌈(n+1)(1−α)⌉/n\lceil (n+1)(1-\alpha) \rceil / n empirical quantile and ⌈501×0.9⌉=451\lceil 501 \times 0.9 \rceil = 451

D

The 452nd smallest score: take ⌈(n+1)(1−α)⌉\lceil (n+1)(1-\alpha) \rceil, then round up once more to leave room for the test point