Learning Optimal Control via Forward and Backward Stochastic Differential Equations

Abstract

In this paper we present a novel sampling-based numerical scheme designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear version of the Feynman-Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Bellman equation, expressed in the form of a decoupled system of FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. To enhance the efficiency of the proposed scheme when treating more complex nonlinear systems, we then derive an iterative modification based on Girsanov's theorem on the change of measure, which features importance sampling. The modified scheme is capable of learning the optimal control without requiring an initial guess. We present simulations that validate the algorithm and demonstrate its efficiency in treating nonlinear dynamics.

References in corpus (1)

• Numerical simulation of quadratic BSDEs

Cited by in corpus (18)

• Learning Deep Stochastic Optimal Control Policies using Forward-Backward SDEs
• Variational approach to rare event simulation using least-squares regression
• Optimal Diffusion Processes
• High-Relative Degree Stochastic Control Lyapunov and Barrier Functions
• Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs
• Deep 2FBSDEs For Systems With Control Multiplicative Noise
• Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory
• NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Control
• Deep Forward-Backward SDEs for Min-max Control
• Deterministic particle flows for constraining SDEs
• Large-Scale Multi-Agent Deep FBSDEs
• State Constrained Stochastic Optimal Control Using LSTMs
• Deep Learning for Constrained Utility Maximisation
• An Effective Discrete Recursive Method for Stochastic Optimal Control Problems
• Deep ${\mathcal{L}}^1$ Stochastic Optimal Control Policies for Planetary Soft-landing
• Open-loop Deterministic Density Control of Marked Jump Diffusions
• Deterministic particle flows for constraining stochastic nonlinear systems
• Discrete-time approximation for stochastic optimal control problems under the $G$-expectation framework