Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems

Abstract

Physics-informed neural networks (PINNs) have recently emerged as an alternative way of solving partial differential equations (PDEs) without the need of building elaborate grids, instead, using a straightforward implementation. In particular, in addition to the deep neural network (DNN) for the solution, a second DNN is considered that represents the residual of the PDE. The residual is then combined with the mismatch in the given data of the solution in order to formulate the loss function. This framework is effective but is lacking uncertainty quantification of the solution due to the inherent randomness in the data or due to the approximation limitations of the DNN architecture. Here, we propose a new method with the objective of endowing the DNN with uncertainty quantification for both sources of uncertainty, i.e., the parametric uncertainty and the approximation uncertainty. We first account for the parametric uncertainty when the parameter in the differential equation is represented as a stochastic process. Multiple DNNs are designed to learn the modal functions of the arbitrary polynomial chaos (aPC) expansion of its solution by using stochastic data from sparse sensors. We can then make predictions from new sensor measurements very efficiently with the trained DNNs. Moreover, we employ dropout to correct the over-fitting and also to quantify the uncertainty of DNNs in approximating the modal functions. We then design an active learning strategy based on the dropout uncertainty to place new sensors in the domain to improve the predictions of DNNs. Several numerical tests are conducted for both the forward and the inverse problems to quantify the effectiveness of PINNs combined with uncertainty quantification. This NN-aPC new paradigm of physics-informed deep learning with uncertainty quantification can be readily applied to other types of stochastic PDEs in multi-dimensions.

References in corpus (29)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• TensorFlow: A system for large-scale machine learning
• Improving neural networks by preventing co-adaptation of feature detectors
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Weight Uncertainty in Neural Networks
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• On the Complexity of Best Arm Identification in Multi-Armed Bandit Models
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
• Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding
• Deep Gaussian Processes
• Variational Bayesian Inference with Stochastic Search
• Dropout Inference in Bayesian Neural Networks with Alpha-divergences
• A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations
• Neural-net-induced Gaussian process regression for function approximation and PDE solution
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations
• Variational Bayesian dropout: pitfalls and fixes
• A data-driven framework for sparsity-enhanced surrogates with arbitrary mutually dependent randomness
• Probabilistic solvers for partial differential equations

Cited by in corpus (106)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• DeepXDE: A deep learning library for solving differential equations
• B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data
• fPINNs: Fractional Physics-Informed Neural Networks
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Deep Autoencoder based Energy Method for the Bending, Vibration, and Buckling Analysis of Kirchhoff Plates
• Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• Physics-Informed Neural Network for Modelling the Thermochemical Curing Process of Composite-Tool Systems During Manufacture
• Stiff-PINN: Physics-Informed Neural Network for Stiff Chemical Kinetics
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks
• A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder
• Non-invasive Inference of Thrombus Material Properties with Physics-informed Neural Networks
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
• A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic
• Distributed physics informed neural network for data-efficient solution to partial differential equations
• An overview on deep learning-based approximation methods for partial differential equations
• Deep Learning for One-dimensional Consolidation
• Solving forward and inverse problems of the logarithmic nonlinear Schrodinger equation with PT-symmetric harmonic potential via deep learning
• Multifidelity Modeling for Physics-Informed Neural Networks (PINNs)
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• Active Training of Physics-Informed Neural Networks to Aggregate and Interpolate Parametric Solutions to the Navier-Stokes Equations
• Stochastic Physics-Informed Neural Ordinary Differential Equations
• Physics-Informed Deep Learning For Traffic State Estimation: A Survey and the Outlook
• PI-VAE: Physics-Informed Variational Auto-Encoder for stochastic differential equations
• Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models
• Algorithmically-Consistent Deep Learning Frameworks for Structural Topology Optimization
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems
• Bayesian Physics-Informed Extreme Learning Machine for Forward and Inverse PDE Problems with Noisy Data
• Deep learning neural networks for the third-order nonlinear Schrodinger equation: Solitons, breathers, and rogue waves
• Bayesian Physics Informed Neural Networks for Data Assimilation and Spatio-Temporal Modelling of Wildfires
• PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics
• GINNs: Graph-Informed Neural Networks for Multiscale Physics
• Learning Specialized Activation Functions for Physics-informed Neural Networks
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Error estimates of residual minimization using neural networks for linear PDEs
• Physics informed deep learning for computational elastodynamics without labeled data
• Efficient nonlinear manifold reduced order model
• The Neural Network Approach to Inverse Problems in Differential Equations
• Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
• Predicting Opinion Dynamics via Sociologically-Informed Neural Networks
• Learning Stochastic Dynamics with Statistics-Informed Neural Network
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
• Uncertainty Quantification in Three Dimensional Natural Convection using Polynomial Chaos Expansion and Deep Neural Networks
• New Trends on the Systems Approach to Modeling SARS-CoV-2 Pandemics in a Globally Connected Planet
• A Dimension-Augmented Physics-Informed Neural Network (DaPINN) with High Level Accuracy and Efficiency
• Image Inversion and Uncertainty Quantification for Constitutive Laws of Pattern Formation
• Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs
• Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series
• Deep Hamiltonian networks based on symplectic integrators
• Physics-informed Information Field Theory for Modeling Physical Systems with Uncertainty Quantification
• Bayesian neural networks for weak solution of PDEs with uncertainty quantification
• Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks
• Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs
• Physics-informed deep learning for flow and deformation in poroelastic media
• Mutual Information for Explainable Deep Learning of Multiscale Systems
• SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition
• Physics-Constrained Bayesian Neural Network for Fluid Flow Reconstruction with Sparse and Noisy Data
• Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEs
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• Physics-informed attention-based neural network for solving non-linear partial differential equations
• Physics Informed Neural Networks for Modeling of 3D Flow-Thermal Problems with Sparse Domain Data
• A Practitioner's Guide to Kolmogorov-Arnold Networks
• The mixed deep energy method for resolving concentration features in finite strain hyperelasticity
• Learning on dynamic statistical manifolds
• Modeling Stochastic Microscopic Traffic Behaviors: a Physics Regularized Gaussian Process Approach
• Learning and Meta-Learning of Stochastic Advection-Diffusion-Reaction Systems from Sparse Measurements
• NeuralPDE: Automating Physics-Informed Neural Networks (PINNs) with Error Approximations
• Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation
• Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications
• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• Physics-informed semantic inpainting: Application to geostatistical modeling
• Prediction of Ultrasonic Guided Wave Propagation in Solid-fluid and their Interface under Uncertainty using Machine Learning
• PCE-PINNs: Physics-Informed Neural Networks for Uncertainty Propagation in Ocean Modeling
• One-shot learning for solution operators of partial differential equations
• Learning Density Distribution of Reachable States for Autonomous Systems
• Thermodynamic Consistent Neural Networks for Learning Material Interfacial Mechanics
• Accelerated Gradient-based Design Optimization Via Differentiable Physics-Informed Neural Operator: A Composites Autoclave Processing Case Study
• A Deep Learning Based Discontinuous Galerkin Method for Hyperbolic Equations with Discontinuous Solutions and Random Uncertainties
• Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
• Hammering at the entropy: A GENERIC-guided approach to learning polymeric rheological constitutive equations using PINNs
• Uncertainty Quantification of Locally Nonlinear Dynamical Systems using Neural Networks
• Clustered active-subspace based local Gaussian Process emulator for high-dimensional and complex computer models
• Time-inversion of spatiotemporal beam dynamics using uncertainty-aware latent evolution reversal
• Sensitivity Analysis of Lift and Drag Coefficients for Flow over Elliptical Cylinders of Arbitrary Aspect Ratio and Angle of Attack using Neural Network
• Reversible and irreversible bracket-based dynamics for deep graph neural networks
• Density Propagation with Characteristics-based Deep Learning
• An artificial neural network approximation for Cauchy inverse problems
• Active learning with physics-informed neural networks for optimal sensor placement in deep tunneling through transversely isotropic elastic rocks
• A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
• Learning To Estimate Regions Of Attraction Of Autonomous Dynamical Systems Using Physics-Informed Neural Networks
• Learning To Solve Differential Equations Across Initial Conditions
• Physics-Coupled Spatio-Temporal Active Learning for Dynamical Systems