Calibration of Model Uncertainty for Dropout Variational Inference
arXiv:2006.11584
Abstract
The model uncertainty obtained by variational Bayesian inference with Monte Carlo dropout is prone to miscalibration. In this paper, different logit scaling methods are extended to dropout variational inference to recalibrate model uncertainty. Expected uncertainty calibration error (UCE) is presented as a metric to measure miscalibration. The effectiveness of recalibration is evaluated on CIFAR-10/100 and SVHN for recent CNN architectures. Experimental results show that logit scaling considerably reduce miscalibration by means of UCE. Well-calibrated uncertainty enables reliable rejection of uncertain predictions and robust detection of out-of-distribution data.
References in corpus (11)
- Auto-Encoding Variational Bayes
- Categorical Reparameterization with Gumbel-Softmax
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Weight Uncertainty in Neural Networks
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Variational Dropout and the Local Reparameterization Trick
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- A Simple Baseline for Bayesian Uncertainty in Deep Learning
- Measuring Calibration in Deep Learning
- Evaluating model calibration in classification