Bayesian posterior approximation with stochastic ensembles
arXiv:2212.08123 · doi:10.1109/CVPR52729.2023.01317
Abstract
We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distributions and trained to approximate the Bayesian posterior with variational inference. We implement stochastic ensembles based on Monte Carlo dropout, DropConnect and a novel non-parametric version of dropout and evaluate them on a toy problem and CIFAR image classification. For both tasks, we test the quality of the posteriors directly against Hamiltonian Monte Carlo simulations. Our results show that stochastic ensembles provide more accurate posterior estimates than other popular baselines for Bayesian inference.
arXiv comment19 pages, CVPR 2023
References in corpus (17)
- A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
- Weight Uncertainty in Neural Networks
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
- Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
- Deep Ensembles: A Loss Landscape Perspective
- A Survey of Uncertainty in Deep Neural Networks
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- Dropout Inference in Bayesian Neural Networks with Alpha-divergences
- What Are Bayesian Neural Network Posteriors Really Like?
- Diversity Matters When Learning From Ensembles
- Repulsive Deep Ensembles are Bayesian
- DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks
- Improving robustness and calibration in ensembles with diversity regularization
- Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping