Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

Abstract

Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}. Quantifying uncertainty is especially critical in real-world settings, which often involve input distributions that are shifted from the training distribution due to a variety of factors including sample bias and non-stationarity. In such settings, well calibrated uncertainty estimates convey information about when a model's output should (or should not) be trusted. Many probabilistic deep learning methods, including Bayesian-and non-Bayesian methods, have been proposed in the literature for quantifying predictive uncertainty, but to our knowledge there has not previously been a rigorous large-scale empirical comparison of these methods under dataset shift. We present a large-scale benchmark of existing state-of-the-art methods on classification problems and investigate the effect of dataset shift on accuracy and calibration. We find that traditional post-hoc calibration does indeed fall short, as do several other previous methods. However, some methods that marginalize over models give surprisingly strong results across a broad spectrum of tasks.

Advances in Neural Information Processing Systems, 2019

References in corpus (17)

• End to End Learning for Self-Driving Cars
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Semi-Supervised Learning with Deep Generative Models
• Training Very Deep Networks
• Weight Uncertainty in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Self-Normalizing Neural Networks
• Variational Dropout and the Local Reparameterization Trick
• Scalable Variational Gaussian Process Classification
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
• Invertible Residual Networks
• Uncertainty in the Variational Information Bottleneck
• A Less Biased Evaluation of Out-of-distribution Sample Detectors

Cited by in corpus (211)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
• Deep Ensembles: A Loss Landscape Perspective
• A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving
• Global canopy height regression and uncertainty estimation from GEDI LIDAR waveforms with deep ensembles
• Measuring Massive Multitask Language Understanding
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• MOPO: Model-based Offline Policy Optimization
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Classifier Calibration: A survey on how to assess and improve predicted class probabilities
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization
• Earthquake magnitude and location estimation from real time seismic waveforms with a transformer network
• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• The transformer earthquake alerting model: A new versatile approach to earthquake early warning
• What Are Bayesian Neural Network Posteriors Really Like?
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• Unsolved Problems in ML Safety
• Calibrating Deep Neural Networks using Focal Loss
• A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications
• Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
• Non-Intrusive Reduced-Order Modeling Using Uncertainty-Aware Deep Neural Networks and Proper Orthogonal Decomposition: Application to Flood Modeling
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
• Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends
• Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
• Parameters Estimation for the Cosmic Microwave Background with Bayesian Neural Networks
• Lightweight Pixel Difference Networks for Efficient Visual Representation Learning
• Measuring Robustness in Deep Learning Based Compressive Sensing
• COMBO: Conservative Offline Model-Based Policy Optimization
• Project Achoo: A Practical Model and Application for COVID-19 Detection from Recordings of Breath, Voice, and Cough
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation
• On the Expressiveness of Approximate Inference in Bayesian Neural Networks
• Machine Learning Explainability for External Stakeholders
• Mapping oil palm density at country scale: An active learning approach
• Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making
• Explainability through uncertainty: Trustworthy decision-making with neural networks
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning
• Uncertainty Aware Training to Improve Deep Learning Model Calibration for Classification of Cardiac MR Images
• Hydra: Preserving Ensemble Diversity for Model Distillation
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
• Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
• Uncertainty in Gradient Boosting via Ensembles
• The Unreasonable Effectiveness of Deep Evidential Regression
• Hyperparameter Ensembles for Robustness and Uncertainty Quantification
• Uncertainty Estimation in Autoregressive Structured Prediction
• Deep Ensemble Analysis for Imaging X-ray Polarimetry
• Bayesian Deep Ensembles via the Neural Tangent Kernel
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Laplace Redux -- Effortless Bayesian Deep Learning
• Uncertainty Quantification Metrics for Deep Regression
• Assaying Out-Of-Distribution Generalization in Transfer Learning
• Assessing the performance of LTE and NLTE synthetic stellar spectra in a machine learning framework
• Efficient Bayesian Uncertainty Estimation for nnU-Net
• Bayesian Neural Network Priors Revisited
• Mitigating Bias in Calibration Error Estimation
• Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference
• All You Need is a Good Functional Prior for Bayesian Deep Learning
• The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
• Data-Driven Selection and Spectral Classification of White Dwarf Stars
• Twin Neural Network Regression
• Reliable Neural Networks for Regression Uncertainty Estimation
• Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks
• Boundary thickness and robustness in learning models
• Offline Reinforcement Learning with Reverse Model-based Imagination
• TMS-Net: A Segmentation Network Coupled With A Run-time Quality Control Method For Robust Cardiac Image Segmentation
• Transferable Calibration with Lower Bias and Variance in Domain Adaptation
• Dropout Q-Functions for Doubly Efficient Reinforcement Learning
• BNNpriors: A library for Bayesian neural network inference with different prior distributions
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Neural Ensemble Search for Uncertainty Estimation and Dataset Shift
• Regression Prior Networks
• Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification
• Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation
• Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection
• Repulsive Deep Ensembles are Bayesian
• Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection?
• Learning under Model Misspecification: Applications to Variational and Ensemble methods
• Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift
• Stochastic-YOLO: Efficient Probabilistic Object Detection under Dataset Shifts
• DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing
• Continual Learning in Recurrent Neural Networks
• Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions
• PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction
• Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
• Insurance pricing with hierarchically structured data: An illustration with a workers' compensation insurance portfolio
• What Do We Mean by Generalization in Federated Learning?
• Fully probabilistic quasar continua predictions near Lyman-α with conditional neural spline flows
• Explaining Predictive Uncertainty by Exposing Second-Order Effects
• Counterfactual Data Augmentation using Locally Factored Dynamics
• Diversity inducing Information Bottleneck in Model Ensembles
• Bayesian Deep Learning via Subnetwork Inference
• EREBA: Black-box Energy Testing of Adaptive Neural Networks
• Federated Mixture of Experts
• Anomalous Example Detection in Deep Learning: A Survey
• Fast Predictive Uncertainty for Classification with Bayesian Deep Networks
• Evaluating Robustness to Context-Sensitive Feature Perturbations of Different Granularities
• Combining Ensembles and Data Augmentation can Harm your Calibration
• Assurance Monitoring of Learning Enabled Cyber-Physical Systems Using Inductive Conformal Prediction based on Distance Learning
• Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
• The missing radial velocities of Gaia: blind predictions for DR3
• Soft Calibration Objectives for Neural Networks
• On-manifold Adversarial Data Augmentation Improves Uncertainty Calibration
• Revisiting the Evaluation of Uncertainty Estimation and Its Application to Explore Model Complexity-Uncertainty Trade-Off
• Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection
• Likelihood Ratios for Out-of-Distribution Detection
• When and How Mixup Improves Calibration
• Generalized Negative Correlation Learning for Deep Ensembling
• Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
• Stationary Activations for Uncertainty Calibration in Deep Learning
• Quizbowl: The Case for Incremental Question Answering
• Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks
• Dangers of Bayesian Model Averaging under Covariate Shift
• Closer Look at the Uncertainty Estimation in Semantic Segmentation under Distributional Shift
• Deep Ensembles from a Bayesian Perspective
• Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning
• Explaining The Efficacy of Counterfactually Augmented Data
• Improving robustness and calibration in ensembles with diversity regularization
• Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
• Ranking over Regression for Bayesian Optimization and Molecule Selection
• Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification
• Vision-Based Uncertainty-Aware Motion Planning based on Probabilistic Semantic Segmentation
• Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
• Selective Classification Can Magnify Disparities Across Groups
• Uncertainty Prediction for Deep Sequential Regression Using Meta Models
• Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation
• Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
• NOMU: Neural Optimization-based Model Uncertainty
• Improving Confidence Estimates for Unfamiliar Examples
• A benchmark study on reliable molecular supervised learning via Bayesian learning
• Unsupervised Calibration under Covariate Shift
• Deep Anti-Regularized Ensembles provide reliable out-of-distribution uncertainty quantification
• Testing Human-Hand Segmentation on In-Distribution and Out-of-Distribution Data in Human-Robot Interactions Using a Deep Ensemble Model
• Efficient Uncertainty Estimation in Spiking Neural Networks via MC-dropout
• Bayesian posterior approximation with stochastic ensembles
• AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification
• Posterior Meta-Replay for Continual Learning
• Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling
• A comprehensive study on the prediction reliability of graph neural networks for virtual screening
• On Calibration of Mixup Training for Deep Neural Networks
• Implicit Generative Modeling for Efficient Exploration
• Learnable Uncertainty under Laplace Approximations
• Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
• Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation
• Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning
• Epistemic Neural Networks
• QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning
• Unsupervised Machine Learning Discovery of Chemical and Physical Transformation Pathways from Imaging Data
• Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit
• Bayesian Neural Networks for Virtual Flow Metering: An Empirical Study
• Increasing Trustworthiness of Deep Neural Networks via Accuracy Monitoring
• Sparse Uncertainty Representation in Deep Learning with Inducing Weights
• On the role of Model Uncertainties in Bayesian Optimization
• Distance Matters For Improving Performance Estimation Under Covariate Shift
• Being a Bit Frequentist Improves Bayesian Neural Networks
• Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
• Robustness via Cross-Domain Ensembles
• Uncertainty-Aware COVID-19 Detection from Imbalanced Sound Data
• Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs
• Beyond the Mean-Field: Structured Deep Gaussian Processes Improve the Predictive Uncertainties
• Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging
• The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration
• Selective Ensembles for Consistent Predictions
• Dark solitons in Bose-Einstein condensates: a dataset for many-body physics research
• Automatic Differentiation Variational Inference with Mixtures
• Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentation
• What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?
• Are Bayesian neural networks intrinsically good at out-of-distribution detection?
• Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition
• On the Calibration and Uncertainty of Neural Learning to Rank Models
• Semi-Supervised Class Discovery
• Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs
• ExCon: Explanation-driven Supervised Contrastive Learning for Image Classification
• On Efficient Uncertainty Estimation for Resource-Constrained Mobile Applications
• PØDA: Prompt-driven Zero-shot Domain Adaptation
• Femtosecond pulse parameter estimation from photoelectron momenta using machine learning
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
• Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantification
• Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation
• Uncertainty Propagation in Node Classification
• Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space
• Towards Robust Active Feature Acquisition
• Test Sample Accuracy Scales with Training Sample Density in Neural Networks
• Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis
• Sequential Multivariate Change Detection with Calibrated and Memoryless False Detection Rates
• Revisiting Explicit Regularization in Neural Networks for Well-Calibrated Predictive Uncertainty
• Sparse MoEs meet Efficient Ensembles
• Temporal mixture ensemble models for intraday volume forecasting in cryptocurrency exchange markets
• Bayesian Attention Modules
• Scalable Bayesian neural networks by layer-wise input augmentation
• A Roadmap for Robust End-to-End Alignment
• Golden Grain: Building a Secure and Decentralized Model Marketplace for MLaaS
• A Survey on Assessing the Generalization Envelope of Deep Neural Networks: Predictive Uncertainty, Out-of-distribution and Adversarial Samples
• A Novel Unsupervised Post-Processing Calibration Method for DNNS with Robustness to Domain Shift
• Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs
• Scene Uncertainty and the Wellington Posterior of Deterministic Image Classifiers
• Decision Theoretic Bootstrapping
• Greedy Bayesian Posterior Approximation with Deep Ensembles
• Test for non-negligible adverse shifts
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
• Neural Bootstrapper
• A Kernel Framework to Quantify a Model's Local Predictive Uncertainty under Data Distributional Shifts
• Multivariate Deep Evidential Regression
• Deep Classifiers with Label Noise Modeling and Distance Awareness
• Learning to Predict Error for MRI Reconstruction
• Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods
• On Deep Neural Network Calibration by Regularization and its Impact on Refinement
• Weighted Ensemble Models Are Strong Continual Learners