Deep neural network approximations for Monte Carlo algorithms

Abstract

Recently, it has been proposed in the literature to employ deep neural networks (DNNs) together with stochastic gradient descent methods to approximate solutions of PDEs. There are also a few results in the literature which prove that DNNs can approximate solutions of certain PDEs without the curse of dimensionality in the sense that the number of real parameters used to describe the DNN grows at most polynomially both in the PDE dimension and the reciprocal of the prescribed approximation accuracy. One key argument in most of these results is, first, to use a Monte Carlo approximation scheme which can approximate the solution of the PDE under consideration at a fixed space-time point without the curse of dimensionality and, thereafter, to prove that DNNs are flexible enough to mimic the behaviour of the used approximation scheme. Having this in mind, one could aim for a general abstract result which shows under suitable assumptions that if a certain function can be approximated by any kind of (Monte Carlo) approximation scheme without the curse of dimensionality, then this function can also be approximated with DNNs without the curse of dimensionality. It is a key contribution of this article to make a first step towards this direction. In particular, the main result of this paper, essentially, shows that if a function can be approximated by means of some suitable discrete approximation scheme without the curse of dimensionality and if there exist DNNs which satisfy certain regularity properties and which approximate this discrete approximation scheme without the curse of dimensionality, then the function itself can also be approximated with DNNs without the curse of dimensionality. As an application of this result we establish that solutions of suitable Kolmogorov PDEs can be approximated with DNNs without the curse of dimensionality.

arXiv comment45 pages

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• PDE-Net: Learning PDEs from Data
• Solving parametric PDE problems with artificial neural networks
• Deep learning observables in computational fluid dynamics
• Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations
• Convergence of the Deep BSDE Method for Coupled FBSDEs
• Deep splitting method for parabolic PDEs
• DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing
• Solving the Kolmogorov PDE by means of deep learning
• Solving high-dimensional optimal stopping problems using deep learning
• A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Neural Networks Trained to Solve Differential Equations Learn General Representations
• Uniform error estimates for artificial neural network approximations for heat equations
• Space-time error estimates for deep neural network approximations for differential equations

Cited by in corpus (4)

• A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• An overview on deep learning-based approximation methods for partial differential equations
• Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems
• Strong overall error analysis for the training of artificial neural networks via random initializations