Evals 0/1

Evals

Production monitoring

Watching a deployed model for covariate shift and concept drift, and trying a replacement on shadow traffic before it serves anyone. A model that passed its evaluation can still degrade as the data changes.

0 / 4 solved

Questions

0/4 solved
Question Tags Difficulty Success
1. Same Inputs, New Rules
data driftconcept drift +1
Medium —
2. What is covariate shift?
Machine LearningProduction Monitoring
Medium —
3. What should be monitored after model deployment?
Machine LearningProduction Monitoring
Medium —
4. The Understudy Model
shadow deploymentcanary deployment +2
Medium —