BNNpriors: A library for Bayesian neural network inference with different prior distributions

Abstract

Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance. However, it remains challenging to choose a good prior distribution over their weights. While isotropic Gaussian priors are often chosen in practice due to their simplicity, they do not reflect our true prior beliefs well and can lead to suboptimal performance. Our new library, BNNpriors, enables state-of-the-art Markov Chain Monte Carlo inference on Bayesian neural networks with a wide range of predefined priors, including heavy-tailed ones, hierarchical ones, and mixture priors. Moreover, it follows a modular approach that eases the design and implementation of new custom priors. It has facilitated foundational discoveries on the nature of the cold posterior effect in Bayesian neural networks and will hopefully catalyze future research as well as practical applications in this area.

arXiv commentAccepted for publication at Software Impacts

References in corpus (11)

• Array Programming with NumPy
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization
• Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning
• Marginal likelihood computation for model selection and hypothesis testing: an extensive review
• Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
• Bayesian Neural Network Priors Revisited
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning

Cited by in corpus (11)

• Repulsive Deep Ensembles are Bayesian
• Data augmentation in Bayesian neural networks and the cold posterior effect
• On Stein Variational Neural Network Ensembles
• Graph Evidential Learning for Anomaly Detection
• Quantum Bayesian Neural Networks
• Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
• TyXe: Pyro-based Bayesian neural nets for Pytorch
• Pathologies in priors and inference for Bayesian transformers
• A Bayesian Approach to Invariant Deep Neural Networks
• Deep Classifiers with Label Noise Modeling and Distance Awareness
• Neural Variational Gradient Descent