Twenty Alerts Per Fraud

A fraud model scores 1,000,000 transactions, of which 1,000 are fraudulent. At the chosen threshold it flags 10,000 transactions and 500 of those are genuinely fraud. Which reading is right?

Precision is 0.05, but ROC-AUC remains the better headline metric because it is invariant to the positive-class rate and therefore comparable across datasets.

Recall is 0.5 and precision is 0.05, so an analyst opens 20 alerts per fraud found. The false positive rate is $9{,}500 / 999{,}000 \approx 0.0095$ , which looks excellent only because the negative class has 999,000 members to divide by.

Precision is 0.5 and the false positive rate is 0.0095, so roughly half the alert queue is real fraud.

Precision is 0.05 and the false positive rate is 0.95, so ROC-AUC would look just as bad as PR-AUC here.