Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations

Abstract

Computing optimal feedback controls for nonlinear systems generally requires solving Hamilton-Jacobi-Bellman (HJB) equations, which are notoriously difficult when the state dimension is large. Existing strategies for high-dimensional problems often rely on specific, restrictive problem structures, or are valid only locally around some nominal trajectory. In this paper, we propose a data-driven method to approximate semi-global solutions to HJB equations for general high-dimensional nonlinear systems and compute candidate optimal feedback controls in real-time. To accomplish this, we model solutions to HJB equations with neural networks (NNs) trained on data generated without discretizing the state space. Training is made more effective and data-efficient by leveraging the known physics of the problem and using the partially-trained NN to aid in adaptive data generation. We demonstrate the effectiveness of our method by learning solutions to HJB equations corresponding to the attitude control of a six-dimensional nonlinear rigid body, and nonlinear systems of dimension up to 30 arising from the stabilization of a Burgers'-type partial differential equation. The trained NNs are then used for real-time feedback control of these systems.

arXiv commentAdded section on validation error computation. Updated convergence test formula and associated results

References in corpus (7)

• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
• On some neural network architectures that can represent viscosity solutions of certain high dimensional Hamilton--Jacobi partial differential equations
• A patchy Dynamic Programming scheme for a class of Hamilton-Jacobi-Bellman equations
• Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations
• Interplanetary Transfers via Deep Representations of the Optimal Policy and/or of the Value Function

Cited by in corpus (12)

• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Finding
• Algorithms of Data Development For Deep Learning and Feedback Design
• Neural Network Optimal Feedback Control with Guaranteed Local Stability
• Offline Supervised Learning V.S. Online Direct Policy Optimization: A Comparative Study and A Unified Training Paradigm for Neural Network-Based Optimal Feedback Control
• An Optimal Control Method to Compute the Most Likely Transition Path for Stochastic Dynamical Systems with Jumps
• Deep neural network approximations for the stable manifolds of the Hamilton-Jacobi-Bellman equations
• Neural network optimal feedback control with enhanced closed loop stability
• Separable approximations of optimal value functions under a decaying sensitivity assumption
• Formally Verified Physics-Informed Neural Control Lyapunov Functions
• Progressive Optimal Path Sampling for Closed-Loop Optimal Control Design with Deep Neural Networks