Metrics & Calibration

Precision/recall/F1, ROC vs PR curves, probability calibration, and nested cross-validation.

0 / 10 solved

Question Tags Difficulty Success

1. 99% and Still Wrong

accuracyclass-imbalance +1

Easy —

2. Why does temperature scaling preserve top-1 predictions?

Machine LearningEvaluation & Validation

Medium —

3. What does probability calibration fix?

Machine LearningEvaluation & Validation

Medium —

4. Twenty Alerts Per Fraud

precision-recallroc-auc +2

Medium —

5. When can AUROC be misleading?

Machine LearningEvaluation & Validation

Medium —

6. Can a lower Brier score prove better calibration?

Machine LearningEvaluation & Validation

Medium —

7. Why can ECE comparisons be unstable?

Machine LearningEvaluation & Validation

Medium —

8. Curve Ball for Rare Classes

roc-curvepr-curve +1

Medium —

9. Why use nested cross-validation?

Machine LearningEvaluation & Validation

Medium —

10. What does nested cross-validation estimate?

Machine LearningEvaluation & Validation

Medium —

Learning Resources

StatQuest: ROC, precision/recall, calibration

Up next

10 of 10 questions here are still unsolved.

Next module Data Leakage & Preprocessing