Scalable Bayesian neural networks by layer-wise input augmentation
arXiv:2010.13498
Abstract
We introduce implicit Bayesian neural networks, a simple and scalable approach for uncertainty representation in deep learning. Standard Bayesian approach to deep learning requires the impractical inference of the posterior distribution over millions of parameters. Instead, we propose to induce a distribution that captures the uncertainty over neural networks by augmenting each layer's inputs with latent variables. We present appropriate input distributions and demonstrate state-of-the-art performance in terms of calibration, robustness and uncertainty characterisation over large-scale, multi-million parameter image classification tasks.
arXiv comment8 pages
References in corpus (9)
- Wide Residual Networks
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
- Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
- A Simple Baseline for Bayesian Uncertainty in Deep Learning
- Functional Variational Bayesian Neural Networks
- The Case for Bayesian Deep Learning
- Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
- Uncertainty Decomposition in Bayesian Neural Networks with Latent Variables