Deep neural network approximations for the stable manifolds of the Hamilton-Jacobi-Bellman equations

Abstract

For an infinite-horizon control problem, the optimal control can be represented by the stable manifold of the characteristic Hamiltonian system of Hamilton-Jacobi-Bellman (HJB) equation in a semiglobal domain. In this paper, we first theoretically prove that if an approximation is sufficiently close to the exact stable manifold of the HJB equation in a certain sense, then the control derived from this approximation stabilizes the system and is nearly optimal. Then, based on the theoretical result, we propose a deep learning algorithm to approximate the stable manifold and compute optimal feedback control numerically. The algorithm relies on adaptive data generation through finding trajectories randomly within the stable manifold. Such kind of algorithm is grid-free basically, making it potentially applicable to a wide range of high-dimensional nonlinear systems. We demonstrate the effectiveness of our method through two examples: stabilizing the Reaction Wheel Pendulums and controlling the parabolic Allen-Cahn equation.

arXiv commentThe manuscript has been revised to be more readable and concise

References in corpus (11)

• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations
• Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
• A patchy Dynamic Programming scheme for a class of Hamilton-Jacobi-Bellman equations
• Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations
• A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Finding
• Scaling and Balancing for High-Performance Computation of Optimal Controls
• Algorithms of Data Development For Deep Learning and Feedback Design
• Neural Network Optimal Feedback Control with Guaranteed Local Stability