Evals
Production monitoring
Watching a deployed model for covariate shift and concept drift, and trying a replacement on shadow traffic before it serves anyone. A model that passed its evaluation can still degrade as the data changes.
Questions
Question Difficulty Success