Confidence-based Estimators for Predictive Performance in Model Monitoring

Abstract

After a machine learning model has been deployed into production, its predictive performance needs to be monitored. Ideally, such monitoring can be carried out by comparing the model's predictions against ground truth labels. For this to be possible, the ground truth labels must be available relatively soon after inference. However, there are many use cases where ground truth labels are available only after a significant delay, or in the worst case, not at all. In such cases, directly monitoring the model's predictive performance is impossible. Recently, novel methods for estimating the predictive performance of a model when ground truth is unavailable have been developed. Many of these methods leverage model confidence or other uncertainty estimates and are experimentally compared against a naive baseline method, namely Average Confidence (AC), which estimates model accuracy as the average of confidence scores for a given set of predictions. However, until now the theoretical properties of the AC method have not been properly explored. In this paper, we try to fill this gap by reviewing the AC method and show that under certain general assumptions, it is an unbiased and consistent estimator of model accuracy with many desirable properties. We also compare this baseline estimator against some more complex estimators empirically and show that in many cases the AC method is able to beat the others, although the comparative quality of the different estimators is heavily case-dependent.

This version corresponds to the final published version in JAIR. The published article is available at [https://doi.org/10.1613/jair.1.16709]

References in corpus (12)

• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
• Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
• Evaluating model calibration in classification
• Calibrating Deep Neural Networks using Focal Loss
• Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probes
• Mandoline: Model Evaluation under Distribution Shift
• Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift
• Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
• MLDemon: Deployment Monitoring for Machine Learning Systems
• Benchmarking Distribution Shift in Tabular Data with TableShift

Cited by in corpus (1)

• Predictive maintenance solution for industrial systems -- an unsupervised approach based on log periodic power law