Optimal Feedback Law Recovery by Gradient-Augmented Sparse Polynomial Regression

Abstract

A sparse regression approach for the computation of high-dimensional optimal feedback laws arising in deterministic nonlinear control is proposed. The approach exploits the control-theoretical link between Hamilton-Jacobi-Bellman PDEs characterizing the value function of the optimal control problems, and first-order optimality conditions via Pontryagin's Maximum Principle. The latter is used as a representation formula to recover the value function and its gradient at arbitrary points in the space-time domain through the solution of a two-point boundary value problem. After generating a dataset consisting of different state-value pairs, a hyperbolic cross polynomial model for the value function is fitted using a LASSO regression. An extended set of low and high-dimensional numerical tests in nonlinear optimal control reveal that enriching the dataset with gradient information reduces the number of training samples, and that the sparse polynomial regression consistently yields a feedback law of lower complexity.

arXiv commentJournal of Machine Learning Research 21(2020): 1-35

References in corpus (7)

• Solving high-dimensional partial differential equations using deep learning
• Mean-Field Optimal Control
• The max-plus finite element method for solving deterministic optimal control problems: basic properties and convergence analysis
• Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations
• Mean Field Limits for Interacting Diffusions in a Two-Scale Potential
• Taylor Expansions of the Value Function Associated with a Bilinear Optimal Control Problem
• A neural network based policy iteration algorithm with global $H^2$-superlinear convergence for stochastic games on domains

Cited by in corpus (4)

• Neural Network Optimal Feedback Control with Guaranteed Local Stability
• Offline Supervised Learning V.S. Online Direct Policy Optimization: A Comparative Study and A Unified Training Paradigm for Neural Network-Based Optimal Feedback Control
• Reliable optimal controls for SEIR models in epidemiology
• Neural network optimal feedback control with enhanced closed loop stability