Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems

Abstract

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than standard feed-forward neural networks, recurrent neural networks, or convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering. In this work, we review such methods as well as their extensions for parametric studies and for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

References in corpus (58)

• Adam: A Method for Stochastic Optimization
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Gaussian Error Linear Units (GELUs)
• DeepXDE: A deep learning library for solving differential equations
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Fourier Neural Operator for Parametric Partial Differential Equations
• fPINNs: Fractional Physics-Informed Neural Networks
• Learned Primal-dual Reconstruction
• Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Weak Adversarial Networks for High-dimensional Partial Differential Equations
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Stable Architectures for Deep Neural Networks
• Deep D-bar: Real time Electrical Impedance Tomography Imaging with Deep Neural Networks
• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• Provable approximation properties for deep neural networks
• Deep Network Approximation for Smooth Functions
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Machine Learning in QM/MM Molecular Dynamics Simulations of Condensed-Phase Systems
• Physics-Informed Neural Operator for Learning Partial Differential Equations
• Deep Learning in Photoacoustic Tomography: Current approaches and future directions
• Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions
• On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations
• Deep splitting method for parabolic PDEs
• The LoDoPaB-CT Dataset: A Benchmark Dataset for Low-Dose CT Reconstruction Methods
• CCSNet: a deep learning modeling suite for CO${}_2$ storage
• Regularization by architecture: A deep prior approach for inverse problems
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Deep learning-based reduced order models in cardiac electrophysiology
• Numerical Solution of Inverse Problems by Weak Adversarial Networks
• Multiwavelet-based Operator Learning for Differential Equations
• Generative Modeling of Turbulence
• Wavelet neural operator: a neural operator for parametric partial differential equations
• Uniform error estimates for artificial neural network approximations for heat equations
• A Meshfree Generalized Finite Difference Method for Solution Mining Processes
• Nonlocality and Nonlinearity Implies Universality in Operator Learning
• Expressivity of Deep Neural Networks
• Multiscale DeepONet for Nonlinear Operators in Oscillatory Function Spaces for Building Seismic Wave Responses
• Deep neural network approximations for Monte Carlo algorithms
• Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEs
• Learning-informed parameter identification in nonlinear time-dependent PDEs
• Convergence rate of DeepONets for learning operators arising from advection-diffusion equations
• A weighted first-order formulation for solving anisotropic diffusion equations with deep neural networks
• Enhanced DeepONet for Modeling Partial Differential Operators Considering Multiple Input Functions
• Operator learning with PCA-Net: upper and lower complexity bounds
• Variational Bayes Deep Operator Network: A data-driven Bayesian solver for parametric differential equations
• Variable-Input Deep Operator Networks
• On the Representation of Solutions to Elliptic PDEs in Barron Spaces
• The Cost-Accuracy Trade-Off In Operator Learning With Neural Networks
• Proof of the Theory-to-Practice Gap in Deep Learning via Sampling Complexity bounds for Neural Network Approximation Spaces
• Space-time deep neural network approximations for high-dimensional partial differential equations
• Parameter Identification by Deep Learning of a Material Model for Granular Media

Cited by in corpus (4)

• DNN-MG: A Hybrid Neural Network/Finite Element Method with Applications to 3D Simulations of the Navier-Stokes Equations
• Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
• Preconditioned FEM-based Neural Networks for Solving Incompressible Fluid Flows and Related Inverse Problems
• The learned range test method for the inverse inclusion problem