Single-Model Uncertainties for Deep Learning

Abstract

We provide single-model estimates of aleatoric and epistemic uncertainty for deep neural networks. To estimate aleatoric uncertainty, we propose Simultaneous Quantile Regression (SQR), a loss function to learn all the conditional quantiles of a given target variable. These quantiles can be used to compute well-calibrated prediction intervals. To estimate epistemic uncertainty, we propose Orthonormal Certificates (OCs), a collection of diverse non-constant functions that map all training samples to zero. These certificates map out-of-distribution examples to non-zero values, signaling epistemic uncertainty. Our uncertainty estimators are computationally attractive, as they do not require ensembling or retraining deep models, and achieve competitive performance.

arXiv commentTo appear in NeurIPS 2019

Cited by in corpus (36)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• Active learning of deep surrogates for PDEs: Application to metasurface design
• Quality of Uncertainty Quantification for Bayesian Neural Network Inference
• Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• DeepGLEAM: A hybrid mechanistic and deep learning model for COVID-19 forecasting
• Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification
• PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction
• A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
• Outlier-Robust Optimal Transport
• Hyperboost: Hyperparameter Optimization by Gradient Boosting surrogate models
• Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles
• PI3NN: Out-of-distribution-aware prediction intervals from three neural networks
• Exploring Uncertainty in Deep Learning for Construction of Prediction Intervals
• Distinguishing Cause from Effect Using Quantiles: Bivariate Quantile Causal Discovery
• Heteroscedastic Calibration of Uncertainty Estimators in Deep Learning
• Should artificial agents ask for help in human-robot collaborative problem-solving?
• Addressing Variance Shrinkage in Variational Autoencoders using Quantile Regression
• Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
• Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems
• Multi-Loss Sub-Ensembles for Accurate Classification with Uncertainty Estimation
• From Predictions to Decisions: Using Lookahead Regularization
• Limits of Probabilistic Safety Guarantees when Considering Human Uncertainty
• Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervals
• The Earth Mover's Pinball Loss: Quantiles for Histogram-Valued Regression
• Implicit Quantile Neural Networks for Jet Simulation and Correction
• Uncertainty-aware INVASE: Enhanced Breast Cancer Diagnosis Feature Selection
• Quantifying Uncertainty in Deep Spatiotemporal Forecasting
• Regularization Strategies for Quantile Regression
• $f$-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception
• Why Calibration Error is Wrong Given Model Uncertainty: Using Posterior Predictive Checks with Deep Learning
• Deep Classifiers with Label Noise Modeling and Distance Awareness
• Uncertainty Surrogates for Deep Learning
• Estimating the Uncertainty of Neural Network Forecasts for Influenza Prevalence Using Web Search Activity
• $\Delta$-UQ: Accurate Uncertainty Quantification via Anchor Marginalization