From self-tuning regulators to reinforcement learning and back again

Abstract

Machine and reinforcement learning (RL) are increasingly being applied to plan and control the behavior of autonomous systems interacting with the physical world. Examples include self-driving vehicles, distributed sensor networks, and agile robots. However, when machine learning is to be applied in these new settings, the algorithms had better come with the same type of reliability, robustness, and safety bounds that are hallmarks of control theory, or failures could be catastrophic. Thus, as learning algorithms are increasingly and more aggressively deployed in safety critical settings, it is imperative that control theorists join the conversation. The goal of this tutorial paper is to provide a starting point for control theorists wishing to work on learning related problems, by covering recent advances bridging learning and control theory, and by placing these results within an appropriate historical context of system identification and adaptive control.

arXiv commentTutorial paper, 2019 IEEE Conference on Decision and Control, to appear

References in corpus (19)

• Continuous control with deep reinforcement learning
• Trust Region Policy Optimization
• On the Complexity of Best Arm Identification in Multi-Armed Bandit Models
• Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator
• Gradient Descent Learns Linear Dynamical Systems
• Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning
• On the Sample Complexity of the Linear Quadratic Regulator
• Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems
• Near optimal finite time identification of arbitrary linear dynamical systems
• Model-Free Linear Quadratic Control via Reduction to Expert Prediction
• Online Regret Bounds for Undiscounted Continuous Reinforcement Learning
• Model-based Reinforcement Learning and the Eluder Dimension
• On the Sample Complexity of Reinforcement Learning with a Generative Model
• Unimodal Bandits: Regret Lower Bounds and Optimal Algorithms
• Exploration in Structured Reinforcement Learning
• Fastest Convergence for Q-learning
• Learning Linear-Quadratic Regulators Efficiently with only $\sqrt{T}$ Regret
• Unimodal Bandits without Smoothness
• System Level Synthesis

Cited by in corpus (7)

• Data-Driven Model Predictive Control with Stability and Robustness Guarantees
• Robust data-driven state-feedback design
• On the Sample Complexity of Stability Constrained Imitation Learning
• A Tutorial on Concentration Bounds for System Identification
• Optimistic robust linear quadratic dual control
• Linear System Identification Under Multiplicative Noise from Multiple Trajectory Data
• Model-Free $\mu$ Synthesis via Adversarial Reinforcement Learning