Distance Matters For Improving Performance Estimation Under Covariate Shift

Abstract

Performance estimation under covariate shift is a crucial component of safe AI model deployment, especially for sensitive use-cases. Recently, several solutions were proposed to tackle this problem, most leveraging model predictions or softmax confidence to derive accuracy estimates. However, under dataset shifts, confidence scores may become ill-calibrated if samples are too far from the training distribution. In this work, we show that taking into account distances of test samples to their expected training distribution can significantly improve performance estimation under covariate shift. Precisely, we introduce a "distance-check" to flag samples that lie too far from the expected distribution, to avoid relying on their untrustworthy model outputs in the accuracy estimation step. We demonstrate the effectiveness of this method on 13 image classification tasks, across a wide-range of natural and synthetic distribution shifts and hundreds of models, with a median relative MAE improvement of 27% over the best baseline across all tasks, and SOTA performance on 10 out of 13 tasks. Our code is publicly available at https://github.com/melanibe/distance_matters_performance_estimation.

arXiv commentAccepted to ICCV Workshop on Uncertainty Quantification for Computer Vision 2023

References in corpus (10)

• MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Learning Robust Global Representations by Penalizing Local Predictive Power
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
• Estimating Generalization under Distribution Shifts via Domain-Invariant Representations
• Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift
• A Note on "Assessing Generalization of SGD via Disagreement"