QRnet: optimal regulator design with LQR-augmented neural networks

Abstract

In this paper we propose a new computational method for designing optimal regulators for high-dimensional nonlinear systems. The proposed approach leverages physics-informed machine learning to solve high-dimensional Hamilton-Jacobi-Bellman equations arising in optimal feedback control. Concretely, we augment linear quadratic regulators with neural networks to handle nonlinearities. We train the augmented models on data generated without discretizing the state space, enabling application to high-dimensional problems. We use the proposed method to design a candidate optimal regulator for an unstable Burgers' equation, and through this example, demonstrate improved robustness and accuracy compared to existing neural network formulations.

arXiv commentAdded IEEE accepted manuscript with copyright notice

References in corpus (10)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• DGM: A deep learning algorithm for solving partial differential equations
• Solving high-dimensional partial differential equations using deep learning
• Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs
• A patchy Dynamic Programming scheme for a class of Hamilton-Jacobi-Bellman equations
• Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations
• Using Neural Networks to Compute Approximate and Guaranteed Feasible Hamilton-Jacobi-Bellman PDE Solutions
• Interplanetary Transfers via Deep Representations of the Optimal Policy and/or of the Value Function
• Tensor Decomposition Methods for High-dimensional Hamilton-Jacobi-Bellman Equations

Cited by in corpus (8)

• Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs
• Neural Network Optimal Feedback Control with Guaranteed Local Stability
• Neural network optimal feedback control with enhanced closed loop stability
• Lax-Oleinik-type formulas and efficient algorithms for certain high-dimensional optimal control problems
• Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems
• Hopf-type representation formulas and efficient algorithms for certain high-dimensional optimal control problems
• Progressive Optimal Path Sampling for Closed-Loop Optimal Control Design with Deep Neural Networks
• Value-Gradient based Formulation of Optimal Control Problem and Machine Learning Algorithm