Well-calibrated Model Uncertainty with Temperature Scaling for Dropout Variational Inference

Abstract

Model uncertainty obtained by variational Bayesian inference with Monte Carlo dropout is prone to miscalibration. The uncertainty does not represent the model error well. In this paper, temperature scaling is extended to dropout variational inference to calibrate model uncertainty. Expected uncertainty calibration error (UCE) is presented as a metric to measure miscalibration of uncertainty. The effectiveness of this approach is evaluated on CIFAR-10/100 for recent CNN architectures. Experimental results show, that temperature scaling considerably reduces miscalibration by means of UCE and enables robust rejection of uncertain predictions. The proposed approach can easily be derived from frequentist temperature scaling and yields well-calibrated model uncertainty. It is simple to implement and does not affect the model accuracy.

Accepted at 4th workshop on Bayesian Deep Learning (NeurIPS 2019)

References in corpus (5)

• Auto-Encoding Variational Bayes
• Decoupled Weight Decay Regularization
• Categorical Reparameterization with Gumbel-Softmax
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• On Calibration of Modern Neural Networks

Cited by in corpus (10)

• TransMorph: Transformer for unsupervised medical image registration
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition
• Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotations
• Multi-Loss Sub-Ensembles for Accurate Classification with Uncertainty Estimation
• Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection
• Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentation
• Uncertainty Intervals for Graph-based Spatio-Temporal Traffic Prediction
• Unsupervised Model Drift Estimation with Batch Normalization Statistics for Dataset Shift Detection and Model Selection
• Normalized Label Distribution: Towards Learning Calibrated, Adaptable and Efficient Activation Maps