Concurrent learning-based approximate optimal regulation

Abstract

In deterministic systems, reinforcement learning-based online approximate optimal control methods typically require a restrictive persistence of excitation (PE) condition for convergence. This paper presents a concurrent learning-based solution to the online approximate optimal regulation problem that eliminates the need for PE. The development is based on the observation that given a model of the system, the Bellman error, which quantifies the deviation of the system Hamiltonian from the optimal Hamiltonian, can be evaluated at any point in the state space. Further, a concurrent learning-based parameter identifier is developed to compensate for parametric uncertainty in the plant dynamics. Uniformly ultimately bounded (UUB) convergence of the system states to the origin, and UUB convergence of the developed policy to the optimal policy are established using a Lyapunov-based analysis, and simulations are performed to demonstrate the performance of the developed controller.

References in corpus (2)

• Approximate Optimal Trajectory Tracking for Continuous Time Nonlinear Systems
• Concurrent learning-based online approximate feedback-Nash equilibrium solution of N-player nonzero-sum differential games

Cited by in corpus (17)

• Concurrent learning for parameter estimation using dynamic state-derivative estimators
• Model-based reinforcement learning for infinite-horizon approximate optimal tracking
• Efficient model-based reinforcement learning for approximate online optimal
• Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions
• Model-based reinforcement learning in differential graphical games
• Online Approximate Optimal Station Keeping of a Marine Craft in the Presence of a Current
• Online Simultaneous State and Parameter Estimation for Second-order Nonlinear Systems
• Lagrangian-based online safe reinforcement learning for state-constrained systems
• Online Output-Feedback Parameter and State Estimation for Second Order Linear Systems
• Output-feedback online optimal control for a class of nonlinear systems
• Memory-Based Data-Driven MRAC Architecture Ensuring Parameter Convergence
• Off-Policy Risk-Sensitive Reinforcement Learning Based Constrained Robust Optimal Control
• Towards Better Adaptive Systems by Combining MAPE, Control Theory, and Machine Learning
• Structured Online Learning-based Control of Continuous-time Nonlinear Systems
• Adaptive Observation-Based Efficient Reinforcement Learning for Uncertain Systems
• Safety aware model-based reinforcement learning for optimal control of a class of output-feedback nonlinear systems
• Reinforcement Learning-based Disturbance Rejection Control for Uncertain Nonlinear Systems