Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation

Abstract

Popular approaches for quantifying predictive uncertainty in deep neural networks often involve distributions over weights or multiple models, for instance via Markov Chain sampling, ensembling, or Monte Carlo dropout. These techniques usually incur overhead by having to train multiple model instances or do not produce very diverse predictions. This comprehensive and extensive survey aims to familiarize the reader with an alternative class of models based on the concept of Evidential Deep Learning: For unfamiliar data, they aim to admit "what they don't know", and fall back onto a prior belief. Furthermore, they allow uncertainty estimation in a single model and forward pass by parameterizing distributions over distributions. This survey recapitulates existing works, focusing on the implementation in a classification setting, before surveying the application of the same paradigm to regression. We also reflect on the strengths and weaknesses compared to other existing methods and provide the most fundamental derivations using a unified notation to aid future research.

References in corpus (50)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• On Calibration of Modern Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Variational Inference with Normalizing Flows
• Weight Uncertainty in Neural Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Pitfalls of Graph Neural Network Evaluation
• Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Evidential Deep Learning to Quantify Classification Uncertainty
• Bayesian Active Learning for Classification and Preference Learning
• Predict then Propagate: Graph Neural Networks meet Personalized PageRank
• Predictive Uncertainty Estimation via Prior Networks
• Deep Ensembles: A Loss Landscape Perspective
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
• Understanding Measures of Uncertainty for Adversarial Example Detection
• Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
• Designing for Responsible Trust in AI Systems: A Communication Perspective
• Measuring Calibration in Deep Learning
• Exploring the Limits of Out-of-Distribution Detection
• What Are Bayesian Neural Network Posteriors Really Like?
• Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
• Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts
• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
• Uncertainty Aware Semi-Supervised Learning on Graph Data
• Understanding the Failure Modes of Out-of-Distribution Generalization
• Understanding Softmax Confidence and Uncertainty
• Reversible Jump MCMC Simulated Annealing for Neural Networks
• The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods
• DEUP: Direct Epistemic Uncertainty Prediction
• Uncertainty on Asynchronous Time Event Prediction
• Regression Prior Networks
• Learning under Model Misspecification: Applications to Variational and Ensemble methods
• Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data
• FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation
• Quantifying Classification Uncertainty using Regularized Evidential Neural Networks
• A Variational Dirichlet Framework for Out-of-Distribution Detection
• Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
• Fast Predictive Uncertainty for Classification with Bayesian Deep Networks
• Linear Connectivity Reveals Generalization Strategies
• Do Bayesian Neural Networks Need To Be Fully Stochastic?
• Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN
• Self-Distribution Distillation: Efficient Uncertainty Estimation

Cited by in corpus (4)

• Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications
• Uncertainty Quantification of Wind Gust Predictions in the Northeast United States: An Evidential Neural Network and Explainable Artificial Intelligence Approach
• Computing a human-like reaction time metric from stable recurrent vision models
• Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantification