Evals 0/1

Evals

Metrics and calibration

Precision and recall, ROC and precision-recall curves, probability calibration and nested cross-validation. A metric that ignores class balance or confidence can rank two models the wrong way round.

0 / 10 solved

Questions

0/10 solved
Question Tags Difficulty Success
1. 99% and Still Wrong
accuracyclass-imbalance +1
Easy —
2. Why does temperature scaling preserve top-1 predictions?
Machine LearningEvaluation & Validation
Medium —
3. What does probability calibration fix?
Machine LearningEvaluation & Validation
Medium —
4. Twenty Alerts Per Fraud
precision-recallroc-auc +2
Medium 50%
5. When can AUROC be misleading?
Machine LearningEvaluation & Validation
Medium —
6. Can a lower Brier score prove better calibration?
Machine LearningEvaluation & Validation
Medium —
7. Why can ECE comparisons be unstable?
Machine LearningEvaluation & Validation
Medium —
8. Curve Ball for Rare Classes
roc-curvepr-curve +1
Medium —
9. Why use nested cross-validation?
Machine LearningEvaluation & Validation
Medium —
10. What does nested cross-validation estimate?
Machine LearningEvaluation & Validation
Medium —