An overview on deep learning-based approximation methods for partial differential equations

Abstract

It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.

49 pages. Compared to the first version, the manuscript has been significantly expanded. In particular, Python source code implementing several of the presented methods using PyTorch, as well as numerical simulations have been added

References in corpus (32)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Fourier Neural Operator for Parametric Partial Differential Equations
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• A regression-based Monte Carlo method to solve backward stochastic differential equations
• Variational Physics-Informed Neural Networks For Solving Partial Differential Equations
• Physics Informed Neural Networks for Simulating Radiative Transfer
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
• Variational Physics Informed Neural Networks: the role of quadratures and test functions
• Stratified regression Monte-Carlo scheme for semilinear PDEs and BSDEs with large scale parallelization on GPUs
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Solving PDEs by Variational Physics-Informed Neural Networks: an a posteriori error analysis
• Numerical simulation of BSDEs using empirical regression methods: theory and practice
• Clifford Neural Layers for PDE Modeling
• MIM: A deep mixed residual method for solving high-order partial differential equations
• Convergence of the Deep BSDE method for FBSDEs with non-Lipschitz coefficients
• Variations on branching methods for non linear PDEs
• Results on numerics for FBSDE with drivers of quadratic growth
• Generalised multilevel Picard approximations
• Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations
• Convergence of Deep Fictitious Play for Stochastic Differential Games
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Multilevel Picard approximations for high-dimensional semilinear second-order PDEs with Lipschitz nonlinearities
• Deep learning based numerical approximation algorithms for stochastic partial differential equations
• Deep Ritz revisited
• High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations
• Higher-order Quasi-Monte Carlo Training of Deep Neural Networks
• Nonlinear Monte Carlo methods with polynomial runtime for high-dimensional iterated nested expectations
• Monte-Carlo Algorithms for Forward Feynman-Kac type representation for semilinear nonconservative Partial Differential Equations

Cited by in corpus (17)

• Enforcing Dirichlet boundary conditions in physics-informed neural networks and variational physics-informed neural networks
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• Numerical methods for backward stochastic differential equations: A survey
• Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics
• Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons
• Neural networks-based algorithms for stochastic control and PDEs in finance
• Physics-informed attention-based neural network for solving non-linear partial differential equations
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Meshfree Variational Physics Informed Neural Networks (MF-VPINN): an adaptive training strategy
• Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality
• Full history recursive multilevel Picard approximations for ordinary differential equations with expectations
• Universal Fourier Neural Operators for periodic homogenization problems in linear elasticity
• Asymptotic-Preserving Neural Networks for Multiscale Time-Dependent Linear Transport Equations
• Simultaneous upper and lower bounds of American-style option prices with hedging via neural networks
• A learning scheme by sparse grids and Picard approximations for semilinear parabolic PDEs
• Strong $L^p$-error analysis of nonlinear Monte Carlo approximations for high-dimensional semilinear partial differential equations
• Neural Networks in Fréchet spaces