Tractable Approximate Gaussian Inference for Bayesian Neural Networks

Abstract

In this paper, we propose an analytical method for performing tractable approximate Gaussian inference (TAGI) in Bayesian neural networks. The method enables the analytical Gaussian inference of the posterior mean vector and diagonal covariance matrix for weights and biases. The method proposed has a computational complexity of $\mathcal{O}(n)$ with respect to the number of parameters $n$, and the tests performed on regression and classification benchmarks confirm that, for a same network architecture, it matches the performance of existing methods relying on gradient backpropagation.

References in corpus (7)

• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Weight Uncertainty in Neural Networks
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• Variational Dropout and the Local Reparameterization Trick
• Practical Deep Learning with Bayesian Principles
• Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors
• A Unifying Bayesian View of Continual Learning

Cited by in corpus (2)

• Analytically Tractable Hidden-States Inference in Bayesian Neural Networks
• Analytically Tractable Inference in Deep Neural Networks