Deep Learning Approximation for Stochastic Control Problems

Abstract

Many real world stochastic control problems suffer from the "curse of dimensionality". To overcome this difficulty, we develop a deep learning approach that directly solves high-dimensional stochastic control problems based on Monte-Carlo sampling. We approximate the time-dependent controls as feedforward neural networks and stack these networks together through model dynamics. The objective function for the control problem plays the role of the loss function for the deep neural network. We test this approach using examples from the areas of optimal trading and energy storage. Our results suggest that the algorithm presented here achieves satisfactory accuracy and at the same time, can handle rather high dimensional problems.

References in corpus (1)

• Benchmarking Deep Reinforcement Learning for Continuous Control

Cited by in corpus (42)

• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problems
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Finding
• Monge-Ampère Flow for Generative Modeling
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• Evolutionary reinforcement learning of dynamical large deviations
• A Neural Network Approach Applied to Multi-Agent Optimal Control
• Multi-scale Deep Neural Networks for Solving High Dimensional PDEs
• Algorithms of Data Development For Deep Learning and Feedback Design
• Neural networks-based algorithms for stochastic control and PDEs in finance
• Neural Network Optimal Feedback Control with Guaranteed Local Stability
• Offline Supervised Learning V.S. Online Direct Policy Optimization: A Comparative Study and A Unified Training Paradigm for Neural Network-Based Optimal Feedback Control
• A neural network based policy iteration algorithm with global $H^2$-superlinear convergence for stochastic games on domains
• Convergence of Deep Fictitious Play for Stochastic Differential Games
• Backward Approximate Dynamic Programming with Hidden Semi-Markov Stochastic Models in Energy Storage Optimization
• Efficient Pricing and Hedging of High Dimensional American Options Using Recurrent Networks
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
• Deep Fictitious Play for Stochastic Differential Games
• Deep 2FBSDEs For Systems With Control Multiplicative Noise
• Incremental Nonlinear Stability Analysis of Stochastic Systems Perturbed by Lévy Noise
• Reinforcement Learning via Parametric Cost Function Approximation for Multistage Stochastic Programming
• Deep Learning Methods for Mean Field Control Problems with Delay
• Signatured Deep Fictitious Play for Mean Field Games with Common Noise
• Swing contract pricing: with and without Neural Networks
• Deep Forward-Backward SDEs for Min-max Control
• Deep Learning for Mean Field Games and Mean Field Control with Applications to Finance
• Deep Learning Based Online Power Control for Large Energy Harvesting Networks
• Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems
• Machine Learning and Control Theory
• Deep neural network for optimal retirement consumption in defined contribution pension system
• Deep Learning for Constrained Utility Maximisation
• Design of A Two-point Steering Path Planner Using Geometric Control
• Deep learning for gradient flows using the Brezis-Ekeland principle
• Meshfree Approximation for Stochastic Optimal Control Problems
• Approximate policy iteration using neural networks for storage problems
• Deep Semi-Martingale Optimal Transport
• A Two-Part Controller Synthesis Approach for Nonlinear Stochastic Systems Perturbed by Lévy Noise Using Renewal Theory and HJB-Based Impulse Control
• Solving stochastic optimal control problem via stochastic maximum principle with deep learning method