Multi-fidelity Bayesian Neural Networks: Algorithms and Applications

Abstract

We propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity, and we apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs). These multi-fidelity BNNs consist of three neural networks: The first is a fully connected neural network, which is trained following the maximum a posteriori probability (MAP) method to fit the low-fidelity data; the second is a Bayesian neural network employed to capture the cross-correlation with uncertainty quantification between the low- and high-fidelity data; and the last one is the physics-informed neural network, which encodes the physical laws described by PDEs. For the training of the last two neural networks, we use the Hamiltonian Monte Carlo method to estimate accurately the posterior distributions for the corresponding hyperparameters. We demonstrate the accuracy of the present method using synthetic data as well as real measurements. Specifically, we first approximate a one- and four-dimensional function, and then infer the reaction rates in one- and two-dimensional diffusion-reaction systems. Moreover, we infer the sea surface temperature (SST) in the Massachusetts and Cape Cod Bays using satellite images and in-situ measurements. Taken together, our results demonstrate that the present method can capture both linear and nonlinear correlation between the low- and high-fideilty data adaptively, identify unknown parameters in PDEs, and quantify uncertainties in predictions, given a few scattered noisy high-fidelity data. Finally, we demonstrate that we can effectively and efficiently reduce the uncertainties and hence enhance the prediction accuracy with an active learning approach, using as examples a specific one-dimensional function approximation and an inverse PDE problem.

31 pages, 11 figures

References in corpus (10)

• MCMC using Hamiltonian dynamics
• Weight Uncertainty in Neural Networks
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Deep Neural Networks as Gaussian Processes
• Stochastic Gradient Hamiltonian Monte Carlo
• Inferring solutions of differential equations using noisy multi-fidelity data
• Learning Properties of Ordered and Disordered Materials from Multi-fidelity Data
• Neural-net-induced Gaussian process regression for function approximation and PDE solution
• Quality of Uncertainty Quantification for Bayesian Neural Network Inference

Cited by in corpus (24)

• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems
• Review of multi-fidelity models
• Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities
• Multi-fidelity surrogate modeling using long short-term memory networks
• Learning Functional Priors and Posteriors from Data and Physics
• Adaptive weighting of Bayesian physics informed neural networks for multitask and multiscale forward and inverse problems
• Bayesian Physics-Informed Extreme Learning Machine for Forward and Inverse PDE Problems with Noisy Data
• Multi-Output Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Uncertainties
• Active Learning with Multifidelity Modeling for Efficient Rare Event Simulation
• A fast multi-fidelity method with uncertainty quantification for complex data correlations: Application to vortex-induced vibrations of marine risers
• A DeepONet multi-fidelity approach for residual learning in reduced order modeling
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• Physics-integrated hybrid framework for model form error identification in nonlinear dynamical systems
• A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling
• Residual Multi-Fidelity Neural Network Computing
• A Bayesian neural network approach to Multi-fidelity surrogate modelling
• PINN-Obs: Physics-Informed Neural Network-Based Observer for Nonlinear Dynamical Systems
• Multi-fidelity Hierarchical Neural Processes
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• Auto-weighted Bayesian Physics-Informed Neural Networks and robust estimations for multitask inverse problems in pore-scale imaging of dissolution
• Physics-informed neural networks for improving cerebral hemodynamics predictions
• Graph Laplacian-based Bayesian Multi-fidelity Modeling
• Flow field tomography with uncertainty quantification using a Bayesian physics-informed neural network