B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data

Abstract

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINN posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

The first two authors contributed equally to this work

References in corpus (20)

• DeepXDE: A deep learning library for solving differential equations
• Weight Uncertainty in Neural Networks
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Deep Neural Networks as Gaussian Processes
• A Conceptual Introduction to Hamiltonian Monte Carlo
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Stochastic Gradient Hamiltonian Monte Carlo
• Inferring solutions of differential equations using noisy multi-fidelity data
• Data-driven discovery of PDEs in complex datasets
• Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems
• Adaptive construction of surrogates for the Bayesian solution of inverse problems
• Neural-net-induced Gaussian process regression for function approximation and PDE solution
• Quality of Uncertainty Quantification for Bayesian Neural Network Inference
• Monge-Ampère Flow for Generative Modeling
• Efficient approximation of high-dimensional functions with neural networks
• Bayesian deep learning with hierarchical prior: Predictions from limited and noisy data
• 'In-Between' Uncertainty in Bayesian Neural Networks
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems

Cited by in corpus (116)

• hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Physics-informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions
• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Machine Learning in Aerodynamic Shape Optimization
• Flow over an espresso cup: Inferring 3D velocity and pressure fields from tomographic background oriented schlieren videos via physics-informed neural networks
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios
• Multi-fidelity Bayesian Neural Networks: Algorithms and Applications
• Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems
• A two-stage physics-informed neural network method based on conserved quantities and applications in localized wave solutions
• Biologically-informed neural networks guide mechanistic modeling from sparse experimental data
• Physics Informed Neural Networks for Simulating Radiative Transfer
• Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
• A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
• Physics-Informed Neural Nets for Control of Dynamical Systems
• Meta-learning PINN loss functions
• When and why PINNs fail to train: A neural tangent kernel perspective
• Deep learning of free boundary and Stefan problems
• Multilevel domain decomposition-based architectures for physics-informed neural networks
• Multifidelity Modeling for Physics-Informed Neural Networks (PINNs)
• Non-Intrusive Reduced-Order Modeling Using Uncertainty-Aware Deep Neural Networks and Proper Orthogonal Decomposition: Application to Flood Modeling
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• Physics-Informed Neural Networks for Nonhomogeneous Material Identification in Elasticity Imaging
• Physics-informed neural network methods based on Miura transformations and discovery of new localized wave solutions
• Enforcing Dirichlet boundary conditions in physics-informed neural networks and variational physics-informed neural networks
• Uncertainty Quantification in Multivariable Regression for Material Property Prediction with Bayesian Neural Networks
• Stochastic Physics-Informed Neural Ordinary Differential Equations
• Physics-informed neural networks method in high-dimensional integrable systems
• Optical Fiber Fault Detection and Localization in a Noisy OTDR Trace Based on Denoising Convolutional Autoencoder and Bidirectional Long Short-Term Memory
• Learning Functional Priors and Posteriors from Data and Physics
• Physics-informed Neural Network Estimation of Material Properties in Soft Tissue Nonlinear Biomechanical Models
• DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimization
• Adaptive weighting of Bayesian physics informed neural networks for multitask and multiscale forward and inverse problems
• Bayesian Physics-Informed Extreme Learning Machine for Forward and Inverse PDE Problems with Noisy Data
• Multi-Output Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Uncertainties
• When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges
• Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors
• Bayesian Physics Informed Neural Networks for Data Assimilation and Spatio-Temporal Modelling of Wildfires
• Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks
• An extended physics informed neural network for preliminary analysis of parametric optimal control problems
• A fast multi-fidelity method with uncertainty quantification for complex data correlations: Application to vortex-induced vibrations of marine risers
• ViscoelasticNet: A physics informed neural network framework for stress discovery and model selection
• Projection pursuit adaptation on polynomial chaos expansions
• A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Physics informed deep learning for computational elastodynamics without labeled data
• HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions
• Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie
• Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performance
• Physics-aware deep neural networks for surrogate modeling of turbulent natural convection
• A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
• Solving time dependent Fokker-Planck equations via temporal normalizing flow
• Laplace HypoPINN: Physics-Informed Neural Network for hypocenter localization and its predictive uncertainty
• Variational Inference at Glacier Scale
• Joint Deep Reversible Regression Model and Physics-Informed Unsupervised Learning for Temperature Field Reconstruction
• I-FENN for thermoelasticity based on physics-informed temporal convolutional network (PI-TCN)
• Learning thermoacoustic interactions in combustors using a physics-informed neural network
• Physics-informed Information Field Theory for Modeling Physical Systems with Uncertainty Quantification
• Bayesian neural networks for weak solution of PDEs with uncertainty quantification
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• L-HYDRA: Multi-Head Physics-Informed Neural Networks
• Spectrally Adapted Physics-Informed Neural Networks for Solving Unbounded Domain Problems
• Improved Surrogate Modeling of Fluid Dynamics with Physics-Informed Neural Networks
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• A hybrid MGA-MSGD ANN training approach for approximate solution of linear elliptic PDEs
• De Rham compatible Deep Neural Network FEM
• Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning
• Physics Informed Neural Networks for Modeling of 3D Flow-Thermal Problems with Sparse Domain Data
• Fast Numerical Simulation of Allen-Cahn Equation
• Physics-Informed Boundary Integral Networks (PIBI-Nets): A Data-Driven Approach for Solving Partial Differential Equations
• Geometry-aware framework for deep energy method: an application to structural mechanics with hyperelastic materials
• Generative prediction of flow fields around an obstacle using the diffusion model
• Meshfree Variational Physics Informed Neural Networks (MF-VPINN): an adaptive training strategy
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling
• Modeling Global Surface Dust Deposition Using Physics-Informed Neural Networks
• Splitting physics-informed neural networks for inferring the dynamics of integer- and fractional-order neuron models
• Scientific machine learning in Hydrology: a unified perspective
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?
• Novel Deep neural networks for solving Bayesian statistical inverse
• Compatible finite element interpolated neural networks
• CDNNs: The coupled deep neural networks for coupling of the Stokes and Darcy-Forchheimer problems
• Multi-variance replica exchange stochastic gradient MCMC for inverse and forward Bayesian physics-informed neural network
• About optimal loss function for training physics-informed neural networks under respecting causality
• Rapidly Encoding Generalizable Dynamics in a Euclidean Symmetric Neural Network
• Auto-weighted Bayesian Physics-Informed Neural Networks and robust estimations for multitask inverse problems in pore-scale imaging of dissolution
• Inverting the Kohn-Sham equations with physics-informed machine learning
• RODE-Net: Learning Ordinary Differential Equations with Randomness from Data
• Applying physics-based loss functions to neural networks for improved generalizability in mechanics problems
• Composed Physics- and Data-driven System Identification for Non-autonomous Systems in Control Engineering
• Bayesian Hidden Physics Models: Uncertainty Quantification for Discovery of Nonlinear Partial Differential Operators from Data
• Spectral Informed Neural Network: An Efficient and Low-Memory PINN
• Banach neural operator for Navier-Stokes equations
• A Physics-Data-Driven Bayesian Method for Heat Conduction Problems
• Physics Informed Neural Networks (PINNs)for approximating nonlinear dispersive PDEs
• Deep learning based on PINN for solving 2 D0F vortex induced vibration of cylinder with high Reynolds number
• Deep learning based on mixed-variable physics informed neural network for solving fluid dynamics without simulation data
• Flow field tomography with uncertainty quantification using a Bayesian physics-informed neural network
• Adjoint-based Particle Forcing Reconstruction and Uncertainty Quantification
• A Machine-Learning Method for Time-Dependent Wave Equations over Unbounded Domains
• Bayesian Reasoning for Physics Informed Neural Networks
• Physics-informed neural networks viewpoint for solving the Dyson-Schwinger equations of quantum electrodynamics
• Unfitted finite element interpolated neural networks
• Uncertainty quantification and stability of neural operators for prediction of three-dimensional turbulence
• Active learning with physics-informed neural networks for optimal sensor placement in deep tunneling through transversely isotropic elastic rocks
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs
• Multi-patch isogeometric neural solver for partial differential equations on computer-aided design domains
• Finite Difference Nets: A Deep Recurrent Framework for Solving Evolution PDEs
• Solver-in-the-Loop joint operator learning: fractional Laplace-Beltrami features for interface reconstruction
• The Machine Learning Approach to Moment Closure Relations for Plasma: A Review
• A Neural Network with Plane Wave Activation for Helmholtz Equation