Looking at the posterior: accuracy and uncertainty of neural-network predictions
arXiv:2211.14605 · doi:10.1088/2632-2153/ad0ab4
Abstract
Bayesian inference can quantify uncertainty in the predictions of neural networks using posterior distributions for model parameters and network output. By looking at these posterior distributions, one can separate the origin of uncertainty into aleatoric and epistemic contributions. One goal of uncertainty quantification is to inform on prediction accuracy. Here we show that prediction accuracy depends on both epistemic and aleatoric uncertainty in an intricate fashion that cannot be understood in terms of marginalized uncertainty distributions alone. How the accuracy relates to epistemic and aleatoric uncertainties depends not only on the model architecture, but also on the properties of the dataset. We discuss the significance of these results for active learning and introduce a novel acquisition function that outperforms common uncertainty-based methods. To arrive at our results, we approximated the posteriors using deep ensembles, for fully-connected, convolutional and attention-based neural networks.
arXiv comment26 pages, 10 figures, 5 tables
References in corpus (18)
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Striving for Simplicity: The All Convolutional Net
- Deep Bayesian Active Learning with Image Data
- Bayesian Active Learning for Classification and Preference Learning
- Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection
- BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
- BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
- Deep learning observables in computational fluid dynamics
- What Are Bayesian Neural Network Posteriors Really Like?
- Large-Scale Visual Active Learning with Deep Probabilistic Ensembles
- Large Scale Structure of Neural Network Loss Landscapes
- Supervised Uncertainty Quantification for Segmentation with Multiple Annotations
- Interpretable, calibrated neural networks for analysis and understanding of inelastic neutron scattering data
- Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
- Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications
- Prediction-Oriented Bayesian Active Learning
- Bayesian posterior approximation with stochastic ensembles