An overview on deep learning-based approximation methods for partial differential equations

Abstract

It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.

arXiv comment49 pages. Compared to the first version, the manuscript has been significantly expanded. In particular, Python source code implementing several of the presented methods using PyTorch, as well as numerical simulations have been added

References in corpus (93)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• DeepXDE: A deep learning library for solving differential equations
• DGM: A deep learning algorithm for solving partial differential equations
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
• Solving high-dimensional partial differential equations using deep learning
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Fourier Neural Operator for Parametric Partial Differential Equations
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Weak Adversarial Networks for High-dimensional Partial Differential Equations
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• A regression-based Monte Carlo method to solve backward stochastic differential equations
• Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving Many-Electron Schrödinger Equation Using Deep Neural Networks
• Approximating quantum many-body wave-functions using artificial neural networks
• A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems
• PDE-Net: Learning PDEs from Data
• Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
• Backflow Transformations via Neural Networks for Quantum Many-Body Wave-Functions
• Deep learning observables in computational fluid dynamics
• Physics Informed Neural Networks for Simulating Radiative Transfer
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometries
• A nonlocal physics-informed deep learning framework using the peridynamic differential operator
• A forward--backward stochastic algorithm for quasi-linear PDEs
• Convergence of the Deep BSDE Method for Coupled FBSDEs
• On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations
• Deep splitting method for parabolic PDEs
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• Deep learning of free boundary and Stefan problems
• Solving Electrical Impedance Tomography with Deep Learning
• A mesh-free method for interface problems using the deep learning approach
• DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing
• Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
• Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach
• Loss of regularity for Kolmogorov equations
• Solving the Kolmogorov PDE by means of deep learning
• Variational Physics Informed Neural Networks: the role of quadratures and test functions
• Simulation of BSDEs by Wiener chaos expansion
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• Multilevel Picard iterations for solving smooth semilinear parabolic heat equations
• A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
• Stratified regression Monte-Carlo scheme for semilinear PDEs and BSDEs with large scale parallelization on GPUs
• Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks
• Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
• Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations
• Neural Networks Trained to Solve Differential Equations Learn General Representations
• Approximation of backward stochastic differential equations using Malliavin weights and least-squares regression
• Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations
• A Discussion on Solving Partial Differential Equations using Neural Networks
• Numerical simulations for full history recursive multilevel Picard approximations for systems of high-dimensional partial differential equations
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• A cubature based algorithm to solve decoupled McKean-Vlasov Forward Backward Stochastic Differential Equations
• On nonlinear Feynman-Kac formulas for viscosity solutions of semilinear parabolic partial differential equations
• Learning Deep Stochastic Optimal Control Policies using Forward-Backward SDEs
• Uniform error estimates for artificial neural network approximations for heat equations
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
• Solving PDEs by Variational Physics-Informed Neural Networks: an a posteriori error analysis
• Numerical simulation of BSDEs using empirical regression methods: theory and practice
• Clifford Neural Layers for PDE Modeling
• MIM: A deep mixed residual method for solving high-order partial differential equations
• Convergence of the Deep BSDE method for FBSDEs with non-Lipschitz coefficients
• Deep neural network approximations for Monte Carlo algorithms
• Variations on branching methods for non linear PDEs
• Generalised multilevel Picard approximations
• Results on numerics for FBSDE with drivers of quadratic growth
• Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations
• Convergence of Deep Fictitious Play for Stochastic Differential Games
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Multilevel Picard approximations for high-dimensional semilinear second-order PDEs with Lipschitz nonlinearities
• Deep learning based numerical approximation algorithms for stochastic partial differential equations
• Monte Carlo for high-dimensional degenerated Semi Linear and Full Non Linear PDEs
• Deep Ritz revisited
• A deep surrogate approach to efficient Bayesian inversion in PDE and integral equation models
• Unbiased deep solvers for linear parametric PDEs
• Deep Curve-dependent PDEs for affine rough volatility
• High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations
• Space-time deep neural network approximations for high-dimensional partial differential equations
• Higher-order Quasi-Monte Carlo Training of Deep Neural Networks
• Nonlinear Monte Carlo methods with polynomial runtime for high-dimensional iterated nested expectations
• Monte-Carlo Algorithms for Forward Feynman-Kac type representation for semilinear nonconservative Partial Differential Equations

Cited by in corpus (18)

• Deep splitting method for parabolic PDEs
• Enforcing Dirichlet boundary conditions in physics-informed neural networks and variational physics-informed neural networks
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• Numerical methods for backward stochastic differential equations: A survey
• Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics
• Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons
• Neural networks-based algorithms for stochastic control and PDEs in finance
• Physics-informed attention-based neural network for solving non-linear partial differential equations
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Meshfree Variational Physics Informed Neural Networks (MF-VPINN): an adaptive training strategy
• Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality
• Full history recursive multilevel Picard approximations for ordinary differential equations with expectations
• Universal Fourier Neural Operators for periodic homogenization problems in linear elasticity
• Asymptotic-Preserving Neural Networks for Multiscale Time-Dependent Linear Transport Equations
• Simultaneous upper and lower bounds of American-style option prices with hedging via neural networks
• A learning scheme by sparse grids and Picard approximations for semilinear parabolic PDEs
• Neural Networks in Fréchet spaces
• Strong $L^p$-error analysis of nonlinear Monte Carlo approximations for high-dimensional semilinear partial differential equations