Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users

Abstract

Modern deep learning methods constitute incredibly powerful tools to tackle a myriad of challenging problems. However, since deep learning methods operate as black boxes, the uncertainty associated with their predictions is often challenging to quantify. Bayesian statistics offer a formalism to understand and quantify the uncertainty associated with deep neural network predictions. This tutorial provides an overview of the relevant literature and a complete toolset to design, implement, train, use and evaluate Bayesian Neural Networks, i.e. Stochastic Artificial Neural Networks trained using Bayesian methods.

20 pages, 13 figures

References in corpus (18)

• Distilling the Knowledge in a Neural Network
• On Calibration of Modern Neural Networks
• Deep Bayesian Active Learning with Image Data
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• A Survey on Deep Learning Techniques for Stereo-based Depth Estimation
• On Markov chain Monte Carlo methods for tall data
• Deep Learning: A Bayesian Perspective
• Dropout Inference in Bayesian Neural Networks with Alpha-divergences
• What Are Bayesian Neural Network Posteriors Really Like?
• Deep Probabilistic Programming
• An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval
• On the Validity of Bayesian Neural Networks for Uncertainty Estimation
• On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
• Why distillation helps: a statistical perspective
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift
• Towards calibrated and scalable uncertainty representations for neural networks
• Prior choice affects ability of Bayesian neural networks to identify unknowns
• On Information Regularization

Cited by in corpus (82)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Improving the open cluster census. II. An all-sky cluster catalogue with Gaia DR3
• Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
• Implicit Full Waveform Inversion with Deep Neural Representation
• A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
• Reliability and Interpretability in Science and Deep Learning
• PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics
• Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation
• Decoding Neutron Star Observations: Revealing Composition through Bayesian Neural Networks
• Farm-wide virtual load monitoring for offshore wind structures via Bayesian neural networks
• Combining data assimilation and machine learning to estimate parameters of a convective-scale model
• Improved uncertainty quantification for neural networks with Bayesian last layer
• Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks
• Low-Order Flow Reconstruction and Uncertainty Quantification in Disturbed Aerodynamics Using Sparse Pressure Measurements
• Bayesian Renormalization
• Machine Learning with High-Cardinality Categorical Features in Actuarial Applications
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Probabilistic Neural Data Fusion for Learning from an Arbitrary Number of Multi-fidelity Data Sets
• NPBDREG: Uncertainty Assessment in Diffeomorphic Brain MRI Registration using a Non-parametric Bayesian Deep-Learning Based Approach
• Bayesian Neural Ordinary Differential Equations
• Real-Time Prediction of Gas Flow Dynamics in Diesel Engines using a Deep Neural Operator Framework
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• From NS observations to nuclear matter properties: a machine learning approach
• DIGS: Deep Inference of Galaxy Spectra with Neural Posterior Estimation
• UMat: Uncertainty-Aware Single Image High Resolution Material Capture
• Physics-informed active learning for accelerating quantum chemical simulations
• Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit
• Uncertainty Aware Neural Network from Similarity and Sensitivity
• Few-shot Quality-Diversity Optimization
• DUDES: Deep Uncertainty Distillation using Ensembles for Semantic Segmentation
• Probabilistic Deep Learning with Probabilistic Neural Networks and Deep Probabilistic Models
• A Bayesian neural network approach to Multi-fidelity surrogate modelling
• The missing radial velocities of Gaia: blind predictions for DR3
• Calibrating Bayesian Generative Machine Learning for Bayesiamplification
• Structured Bayesian Compression for Deep Neural Networks Based on The Turbo-VBI Approach
• PI3NN: Out-of-distribution-aware prediction intervals from three neural networks
• Deep Ensembles from a Bayesian Perspective
• Bayesian Neural Networks: Essentials
• Inferring the Langevin Equation with Uncertainty via Bayesian Neural Networks
• Sparse Bayesian Networks: Efficient Uncertainty Quantification in Medical Image Analysis
• Gibbs Sampling the Posterior of Neural Networks
• Towards Arbitrary QUBO Optimization: Analysis of Classical and Quantum-Activated Feedforward Neural Networks
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• The missing radial velocities of Gaia: a catalogue of Bayesian estimates for DR3
• Bayesian posterior approximation with stochastic ensembles
• Graceful Degradation and Related Fields
• Comparison of Bayesian inference methods using the Loreli II database of hydro-radiative simulations of the 21-cm signal
• A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks
• Principled Pruning of Bayesian Neural Networks through Variational Free Energy Minimization
• Robust quantum dots charge autotuning using neural network uncertainty
• Probabilistic partition of unity networks: clustering based deep approximation
• Evaluating the Quality of the Quantified Uncertainty for (Re)Calibration of Data-Driven Regression Models
• Unifying supervised learning and VAEs -- coverage, systematics and goodness-of-fit in normalizing-flow based neural network models for astro-particle reconstructions
• Bayes2IMC: In-Memory Computing for Bayesian Binary Neural Networks
• Bayesian RG Flow in Neural Network Field Theories
• Estimation of redshift and associated uncertainty of Fermi/LAT extra-galactic sources with Deep Learning
• Uncertainties of Satellite-based Essential Climate Variables from Deep Learning
• Weight fluctuations in (deep) linear neural networks and a derivation of the inverse-variance flatness relation
• From ANN to BNN: Inferring Reionization Parameters using Uncertainty-aware Emulators of 21-cm Summaries
• Density Uncertainty Quantification with NeRF-Ensembles: Impact of Data and Scene Constraints
• In Search of Global 21-cm Signal using Artificial Neural Network in light of ARCADE 2
• Soft Expectation and Deep Maximization for Image Feature Detection
• DnD Filter: Differentiable State Estimation for Dynamic Systems using Diffusion Models
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
• Deep inference of simulated strong lenses in ground-based surveys
• Noise Balance and Stationary Distribution of Stochastic Gradient Descent
• Physics-Informed Machine Learning for Transformer Condition Monitoring -- Part II: Physics-Informed Neural Networks and Uncertainty Quantification
• Building Continuous Quantum-Classical Bayesian Neural Networks for a Classical Clinical Dataset
• Noisy Training Improves E2E ASR for the Edge
• Incorporating Data Uncertainty in Object Tracking Algorithms
• Bayesian classification of astronomical spectra with class uncertainties
• Classification of Fermi-LAT blazars with Bayesian neural networks
• Structured Stochastic Gradient MCMC
• Comparison of Graphcore IPUs and Nvidia GPUsfor cosmology applications
• SEF: A Method for Computing Prediction Intervals by Shifting the Error Function in Neural Networks
• Extraction of the color dipole amplitude with physics-informed neural networks
• A Tutorial on Bayesian Analysis of Linear Shock Compression Data
• Mind the Jumps: A Scalable Robust Local Gaussian Process for Multidimensional Response Surfaces with Discontinuities