Learning Deep Stochastic Optimal Control Policies using Forward-Backward SDEs

Abstract

In this paper we propose a new methodology for decision-making under uncertainty using recent advancements in the areas of nonlinear stochastic optimal control theory, applied mathematics, and machine learning. Grounded on the fundamental relation between certain nonlinear partial differential equations and forward-backward stochastic differential equations, we develop a control framework that is scalable and applicable to general classes of stochastic systems and decision-making problem formulations in robotics and autonomy. The proposed deep neural network architectures for stochastic control consist of recurrent and fully connected layers. The performance and scalability of the aforementioned algorithm are investigated in three non-linear systems in simulation with and without control constraints. We conclude with a discussion on future directions and their implications to robotics.

References in corpus (3)

• Solving high-dimensional partial differential equations using deep learning
• Differentiable MPC for End-to-end Planning and Control
• Deep Dynamical Modeling and Control of Unsteady Fluid Flows

Cited by in corpus (14)

• An overview on deep learning-based approximation methods for partial differential equations
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• High-Relative Degree Stochastic Control Lyapunov and Barrier Functions
• Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs
• Value Iteration in Continuous Actions, States and Time
• Deep 2FBSDEs For Systems With Control Multiplicative Noise
• Learning Locomotion Controllers for Walking Using Deep FBSDE
• NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Control
• Stochastic optimization for learning quantum state feedback control
• State Constrained Stochastic Optimal Control Using LSTMs
• Open-loop Deterministic Density Control of Marked Jump Diffusions
• Large-Scale Multi-Agent Deep FBSDEs
• Deep ${\mathcal{L}}^1$ Stochastic Optimal Control Policies for Planetary Soft-landing