Bayesian Neural Networks: An Introduction and Survey

Abstract

Neural Networks (NNs) have provided state-of-the-art results for many challenging machine learning tasks such as detection, regression and classification across the domains of computer vision, speech recognition and natural language processing. Despite their success, they are often implemented in a frequentist scheme, meaning they are unable to reason about uncertainty in their predictions. This article introduces Bayesian Neural Networks (BNNs) and the seminal research regarding their implementation. Different approximate inference methods are compared, and used to highlight where future research can improve on current methods.

44 pages, 8 figures, Fix typos in Eqn 30, 48, and alpha divergence

References in corpus (4)

• Variational Inference: A Review for Statisticians
• MCMC using Hamiltonian dynamics
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition

Cited by in corpus (18)

• Improving the open cluster census. II. An all-sky cluster catalogue with Gaia DR3
• Probabilistic neural networks for fluid flow surrogate modeling and data recovery
• The DAWES review 10: The impact of deep learning for the analysis of galaxy surveys
• Valid prediction intervals for regression problems
• CONNECT: A neural network based framework for emulating cosmological observables and cosmological parameter inference
• Bayesian Physics-Informed Neural Networks for the Subsurface Tomography based on the Eikonal Equation
• Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks
• Advancing Molecular Machine Learning Representations with Stereoelectronics-Infused Molecular Graphs
• Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks
• Calibrating Bayesian Generative Machine Learning for Bayesiamplification
• Digital Twin-Assisted Robust and Adaptive Resource Slicing in LEO Satellite Networks
• Unified Uncertainty Estimation for Cognitive Diagnosis Models
• An Investigation on Machine Learning Predictive Accuracy Improvement and Uncertainty Reduction using VAE-based Data Augmentation
• Emission Line Predictions for Mock Galaxy Catalogues: a New Differentiable and Empirical Mapping from DESI
• Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference
• The missing radial velocities of Gaia: a catalogue of Bayesian estimates for DR3
• Bayesian and Convolutional Networks for Hierarchical Morphological Classification of Galaxies
• Bayesian classification of astronomical spectra with class uncertainties