Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis

Abstract

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the value function by Monte Carlo regression. This is achieved in the dynamic programming recursion by performance or hybrid iteration, and regress now methods from numerical probabilities. We provide a theoretical justification of these algorithms. Consistency and rate of convergence for the control and value function estimates are analyzed and expressed in terms of the universal approximation error of the neural networks, and of the statistical error when estimating network function, leaving aside the optimization error. Numerical results on various applications are presented in a companion paper (arxiv.org/abs/1812.05916) and illustrate the performance of the proposed algorithms.

To appear in SIAM Journal on Numerical Analysis

References in corpus (3)

• Convergence of the Deep BSDE Method for Coupled FBSDEs
• Deep Learning Approximation for Stochastic Control Problems
• Full error analysis for the training of deep neural networks

Cited by in corpus (21)

• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• A generative adversarial network approach to calibration of local stochastic volatility models
• Full error analysis for the training of deep neural networks
• Variational approach to rare event simulation using least-squares regression
• Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications
• Neural networks-based algorithms for stochastic control and PDEs in finance
• A neural network based policy iteration algorithm with global $H^2$-superlinear convergence for stochastic games on domains
• Schrödinger Bridge Samplers
• Robust utility maximization under model uncertainty via a penalization approach
• Deep Fictitious Play for Stochastic Differential Games
• Risk management with machine-learning-based algorithms
• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• Lax-Oleinik-type formulas and efficient algorithms for certain high-dimensional optimal control problems
• Automatic model training under restrictive time constraints
• Stochastic near-optimal control: additive, multiplicative, non-Markovian and applications
• Simultaneous upper and lower bounds of American-style option prices with hedging via neural networks
• Deep neural network for optimal retirement consumption in defined contribution pension system
• Hopf-type representation formulas and efficient algorithms for certain high-dimensional optimal control problems
• Deep Semi-Martingale Optimal Transport
• Solving stochastic optimal control problem via stochastic maximum principle with deep learning method
• Deep Learning for Constrained Utility Maximisation