Looking at the posterior: accuracy and uncertainty of neural-network predictions
arXiv:2211.14605 · doi:10.1088/2632-2153/ad0ab4
Abstract
Bayesian inference can quantify uncertainty in the predictions of neural networks using posterior distributions for model parameters and network output. By looking at these posterior distributions, one can separate the origin of uncertainty into aleatoric and epistemic contributions. One goal of uncertainty quantification is to inform on prediction accuracy. Here we show that prediction accuracy depends on both epistemic and aleatoric uncertainty in an intricate fashion that cannot be understood in terms of marginalized uncertainty distributions alone. How the accuracy relates to epistemic and aleatoric uncertainties depends not only on the model architecture, but also on the properties of the dataset. We discuss the significance of these results for active learning and introduce a novel acquisition function that outperforms common uncertainty-based methods. To arrive at our results, we approximated the posteriors using deep ensembles, for fully-connected, convolutional and attention-based neural networks.
26 pages, 10 figures, 5 tables
References in corpus (11)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Striving for Simplicity: The All Convolutional Net
- Deep Bayesian Active Learning with Image Data
- Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection
- BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
- What Are Bayesian Neural Network Posteriors Really Like?
- Large Scale Structure of Neural Network Loss Landscapes
- Interpretable, calibrated neural networks for analysis and understanding of inelastic neutron scattering data
- Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
- Prediction-Oriented Bayesian Active Learning
- Bayesian posterior approximation with stochastic ensembles