Models degrade. Cameras get bumped, lighting changes, packaging is redesigned, seasons turn. You do not need a monitoring platform to catch most of it — three checks in your existing alerting will cover the common cases.
Check one: prediction distribution
Record how often each class is predicted, per day. If a defect class that normally appears twelve times a shift drops to zero for two days, something changed — the line, the camera or the model. Alert on the shift, not on the absolute number.
Check two: confidence spread
Track the average confidence of accepted predictions. A slow slide downward is a reliable early warning, usually visible weeks before accuracy complaints reach you.
Check three: a small fixed audit sample
Have an operator label twenty random frames a week. It costs minutes and gives you an actual measured accuracy trend instead of an inferred one. This is the single most useful check on the list.
- Prediction counts per class, per day — alert on a large shift
- Mean confidence of accepted predictions — alert on a downward trend
- Twenty human-labelled frames per week — track measured accuracy
- Input health: frame rate, brightness, blur — catches camera problems first
Watch the input before the output
Most degradation is not the model at all. It is a lens that needs cleaning, a camera that shifted, or a light that failed. Monitoring average frame brightness and blur catches these faster and cheaper than any drift metric.
Write down what happens when an alert fires, and who does it. A drift alert with no owner is a dashboard nobody opens.
Decide the retraining trigger in advance
Retrain on a defined condition — measured accuracy below the agreed threshold for two consecutive weeks — rather than on whoever complains loudest. It keeps the decision boring, which is what you want.