Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks

Abstract

Stochastic partial differential equations (SPDEs) are ubiquitous in engineering and computational sciences. The stochasticity arises as a consequence of uncertainty in input parameters, constitutive relations, initial/boundary conditions, etc. Because of these functional uncertainties, the stochastic parameter space is often high-dimensional, requiring hundreds, or even thousands, of parameters to describe it. This poses an insurmountable challenge to response surface modeling since the number of forward model evaluations needed to construct an accurate surrogate grows exponentially with the dimension of the uncertain parameter space; a phenomenon referred to as the \textit{curse of dimensionality}. State-of-the-art methods for high-dimensional uncertainty propagation seek to alleviate the curse of dimensionality by performing dimensionality reduction in the uncertain parameter space. However, one still needs to perform forward model evaluations that potentially carry a very high computational burden. We propose a novel methodology for high-dimensional uncertainty propagation of elliptic SPDEs which lifts the requirement for a deterministic forward solver. Our approach is as follows. We parameterize the solution of the elliptic SPDE using a deep residual network (ResNet). In a departure from the traditional squared residual (SR) based loss function for training the ResNet, we introduce a novel physics-informed loss function derived from variational principles. Specifically, our loss function is the expectation of the energy functional of the PDE over the stochastic variables. We demonstrate our solver-free approach through various examples where the elliptic SPDE is subjected to different types of high-dimensional input uncertainties. Also, we solve high-dimensional uncertainty propagation and inverse problems.

arXiv comment63 pages, 32 figures

References in corpus (25)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• TensorFlow: A system for large-scale machine learning
• ADADELTA: An Adaptive Learning Rate Method
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Searching for Activation Functions
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• Active subspace methods in theory and practice: applications to kriging surfaces
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• Training-image based geostatistical inversion using a spatial generative adversarial neural network
• Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Exploiting Active Subspaces to Quantify Uncertainty in the Numerical Simulation of the HyShot II Scramjet
• A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equations
• Langevin diffusions and the Metropolis-adjusted Langevin algorithm
• Accelerating MCMC with active subspaces
• Combined parameter and model reduction of cardiovascular problems by means of active subspaces and POD-Galerkin methods
• Computing active subspaces with Monte Carlo
• Higher order Quasi-Monte Carlo integration for Bayesian Estimation
• On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding

Cited by in corpus (40)

• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• A mesh-free method for interface problems using the deep learning approach
• An overview on deep learning-based approximation methods for partial differential equations
• Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network
• Physics-informed neural networks for solving parametric magnetostatic problems
• Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems
• Weak Form Theory-guided Neural Network (TgNN-wf) for Deep Learning of Subsurface Single and Two-phase Flow
• Surrogate and inverse modeling for two-phase flow in porous media via theory-guided convolutional neural network
• Pre-training strategy for solving evolution equations based on physics-informed neural networks
• A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
• On generalized residue network for deep learning of unknown dynamical systems
• A neural network multigrid solver for the Navier-Stokes equations
• GINNs: Graph-Informed Neural Networks for Multiscale Physics
• CEENs: Causality-enforced evolutional networks for solving time-dependent partial differential equations
• Extended Dynamic Mode Decomposition for Inhomogeneous Problems
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• Uncertainty Quantification in Three Dimensional Natural Convection using Polynomial Chaos Expansion and Deep Neural Networks
• A hybrid data driven-physics constrained Gaussian process regression framework with deep kernel for uncertainty quantification
• Unveiling the frontiers of deep learning: innovations shaping diverse domains
• Comparison of Neural FEM and Neural Operator Methods for applications in Solid Mechanics
• Multiple scattering simulation via physics-informed neural networks
• Fast Numerical Simulation of Allen-Cahn Equation
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• Solving Elliptic Equations with Brownian Motion: Bias Reduction and Temporal Difference Learning
• Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides
• Hierarchical Learning to Solve Partial Differential Equations Using Physics-Informed Neural Networks
• Neural Network Solutions to Differential Equations in Non-Convex Domains: Solving the Electric Field in the Slit-Well Microfluidic Device
• Physics-Informed Neural Networks for Solving Multiscale Mode-Resolved Phonon Boltzmann Transport Equation
• Studying First Passage Problems using Neural Networks: A Case Study in the Slit-Well Microfluidic Device
• Sequential bi-level regularized inversion with application to hidden reaction law discovery
• Level set learning with pseudo-reversible neural networks for nonlinear dimension reduction in function approximation
• Recurrent convolutional neural network for the surrogate modeling of subsurface flow simulation
• AutoSurrogate: An LLM-Driven Multi-Agent Framework for Autonomous Construction of Deep Learning Surrogate Models in Subsurface Flow