Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning

Abstract

In recent years, tremendous progress has been made on numerical algorithms for solving partial differential equations (PDEs) in a very high dimension, using ideas from either nonlinear (multilevel) Monte Carlo or deep learning. They are potentially free of the curse of dimensionality for many different applications and have been proven to be so in the case of some nonlinear Monte Carlo methods for nonlinear parabolic PDEs. In this paper, we review these numerical and theoretical advances. In addition to algorithms based on stochastic reformulations of the original problem, such as the multilevel Picard iteration and the Deep BSDE method, we also discuss algorithms based on the more traditional Ritz, Galerkin, and least square formulations. We hope to demonstrate to the reader that studying PDEs as well as control and variational problems in very high dimensions might very well be among the most promising new directions in mathematics and scientific computing in the near future.

References in corpus (75)

• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Solving high-dimensional partial differential equations using deep learning
• DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Deep neural network solution of the electronic Schrödinger equation
• Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks
• Weak Adversarial Networks for High-dimensional Partial Differential Equations
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• A regression-based Monte Carlo method to solve backward stochastic differential equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving Many-Electron Schrödinger Equation Using Deep Neural Networks
• Approximating quantum many-body wave-functions using artificial neural networks
• A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems
• Solving parametric PDE problems with artificial neural networks
• Backflow Transformations via Neural Networks for Quantum Many-Body Wave-Functions
• Deep learning observables in computational fluid dynamics
• A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations
• A forward--backward stochastic algorithm for quasi-linear PDEs
• Convergence of the Deep BSDE Method for Coupled FBSDEs
• On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations
• Deep splitting method for parabolic PDEs
• Deep Learning Approximation for Stochastic Control Problems
• Solving Electrical Impedance Tomography with Deep Learning
• Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations
• DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing
• Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Maximum Principle Based Algorithms for Deep Learning
• Simulation of BSDEs by Wiener chaos expansion
• A Mean-Field Optimal Control Formulation of Deep Learning
• Multilevel Picard iterations for solving smooth semilinear parabolic heat equations
• Pricing and hedging American-style options with deep learning
• Solving high-dimensional optimal stopping problems using deep learning
• A monotone scheme for high-dimensional fully nonlinear PDEs
• Stratified regression Monte-Carlo scheme for semilinear PDEs and BSDEs with large scale parallelization on GPUs
• Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations
• Overcoming the curse of dimensionality in the approximative pricing of financial derivatives with default risks
• Machine Learning from a Continuous Viewpoint
• A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Neural Networks Trained to Solve Differential Equations Learn General Representations
• Overcoming the curse of dimensionality in the numerical approximation of Allen-Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations
• Approximation of backward stochastic differential equations using Malliavin weights and least-squares regression
• Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations
• A Discussion on Solving Partial Differential Equations using Neural Networks
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• Numerical simulations for full history recursive multilevel Picard approximations for systems of high-dimensional partial differential equations
• On nonlinear Feynman-Kac formulas for viscosity solutions of semilinear parabolic partial differential equations
• Uniform error estimates for artificial neural network approximations for heat equations
• Overcoming the curse of dimensionality in the numerical approximation of high-dimensional semilinear elliptic partial differential equations
• Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
• A primal-dual algorithm for BSDEs
• Numerical simulation of BSDEs using empirical regression methods: theory and practice
• Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities
• Algorithms of Data Development For Deep Learning and Feedback Design
• Variations on branching methods for non linear PDEs
• Generalised multilevel Picard approximations
• Convergence of Deep Fictitious Play for Stochastic Differential Games
• Solving Inverse Wave Scattering with Deep Learning
• Multilevel Picard approximations for high-dimensional semilinear second-order PDEs with Lipschitz nonlinearities
• Monte Carlo for high-dimensional degenerated Semi Linear and Full Non Linear PDEs
• Deep Learning-Based BSDE Solver for Libor Market Model with Application to Bermudan Swaption Pricing and Hedging
• Deep Ritz revisited
• Optimal Policies for a Pandemic: A Stochastic Game Approach and a Deep Learning Algorithm
• Deep Curve-dependent PDEs for affine rough volatility
• Neural ODE control for classification, approximation and transport
• Space-time deep neural network approximations for high-dimensional partial differential equations
• FBSDE based Neural Network Algorithms for High-Dimensional Quasilinear Parabolic PDEs
• Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: II -- The Finite Horizon Case
• A Dual Method For Backward Stochastic Differential Equations with Application to Risk Valuation

Cited by in corpus (30)

• An overview on deep learning-based approximation methods for partial differential equations
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Numerical methods for backward stochastic differential equations: A survey
• MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
• Deep learning probability flows and entropy production rates in active matter
• Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities
• Data-driven Identification of 2D Partial Differential Equations using extracted physical features
• Multiscale simulations for multi-continuum Richards equations
• Machine learning based data-driven discovery of nonlinear phase-field dynamics
• Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes
• Imaging Conductivity from Current Density Magnitude using Neural Networks
• SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition
• A model-data asymptotic-preserving neural network method based on micro-macro decomposition for gray radiative transfer equations
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Deep neural network approximations for the stable manifolds of the Hamilton-Jacobi-Bellman equations
• Deep learning based numerical approximation algorithms for stochastic partial differential equations
• Calibration of Local-Stochastic Volatility Models by Optimal Transport
• A Deep Learning Approach for Predicting Spatiotemporal Dynamics From Sparsely Observed Data
• Solving path dependent PDEs with LSTM networks and path signatures
• A Matrix Product State Model for Simultaneous Classification and Generation
• Learning Density Distribution of Reachable States for Autonomous Systems
• Data-informed Deep Optimization
• Full history recursive multilevel Picard approximations for ordinary differential equations with expectations
• Statistics for stochastic differential equations and approximations of resolvent
• A learning scheme by sparse grids and Picard approximations for semilinear parabolic PDEs
• Adaptive feature capture method for solving partial differential equations with near singular solutions
• Risk-indifference Pricing of American-style Contingent Claims
• Deep Semi-Martingale Optimal Transport