Operational Calibration: Debugging Confidence Errors for DNNs in the Field

Abstract

Trained DNN models are increasingly adopted as integral parts of software systems, but they often perform deficiently in the field. A particularly damaging problem is that DNN models often give false predictions with high confidence, due to the unavoidable slight divergences between operation data and training data. To minimize the loss caused by inaccurate confidence, operational calibration, i.e., calibrating the confidence function of a DNN classifier against its operation domain, becomes a necessary debugging step in the engineering of the whole system. Operational calibration is difficult considering the limited budget of labeling operation data and the weak interpretability of DNN models. We propose a Bayesian approach to operational calibration that gradually corrects the confidence given by the model under calibration with a small number of labeled operation data deliberately selected from a larger set of unlabeled operation data. The approach is made effective and efficient by leveraging the locality of the learned representation of the DNN model and modeling the calibration as Gaussian Process Regression. Comprehensive experiments with various practical datasets and DNN models show that it significantly outperformed alternative methods, and in some difficult tasks it eliminated about 71% to 97% high-confidence (>0.9) errors with only about 10\% of the minimal amount of labeled operation data needed for practical learning techniques to barely work.

arXiv commentPublished in the Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020)

References in corpus (23)

• Scikit-learn: Machine Learning in Python
• Distilling the Knowledge in a Neural Network
• End to End Learning for Self-Driving Cars
• Federated Learning: Strategies for Improving Communication Efficiency
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Thumbs up? Sentiment Classification using Machine Learning Techniques
• On Calibration of Modern Neural Networks
• Confounding variables can degrade generalization performance of radiological deep learning models
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• A tutorial on conformal prediction
• DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
• Evidential Deep Learning to Quantify Classification Uncertainty
• Detecting Cancer Metastases on Gigapixel Pathology Images
• Guiding Deep Learning System Testing using Surprise Adequacy
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Accurate Uncertainties for Deep Learning Using Calibrated Regression
• Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Machine Learning Testing: Survey, Landscapes and Horizons
• Can You Trust This Prediction? Auditing Pointwise Reliability After Learning
• Out-of-Distribution Detection using Multiple Semantic Label Representations
• Debugging Machine Learning Tasks

Cited by in corpus (1)

• PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing