Adaptive Regret for Control of Time-Varying Dynamics

Abstract

We consider the problem of online control of systems with time-varying linear dynamics. This is a general formulation that is motivated by the use of local linearization in control of nonlinear dynamical systems. To state meaningful guarantees over changing environments, we introduce the metric of {\it adaptive regret} to the field of control. This metric, originally studied in online learning, measures performance in terms of regret against the best policy in hindsight on {\it any interval in time}, and thus captures the adaptation of the controller to changing dynamics. Our main contribution is a novel efficient meta-algorithm: it converts a controller with sublinear regret bounds into one with sublinear {\it adaptive regret} bounds in the setting of time-varying linear dynamical systems. The main technical innovation is the first adaptive regret bound for the more general framework of online convex optimization with memory. Furthermore, we give a lower bound showing that our attained adaptive regret bound is nearly tight for this general framework.

References in corpus (9)

• Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator
• Improper Learning for Non-Stochastic Control
• Online Control with Adversarial Disturbances
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems
• Logarithmic Regret for Online Control
• Model-Free Linear Quadratic Control via Reduction to Expert Prediction
• Adaptive Regret of Convex and Smooth Functions
• Regret Minimization in Partially Observable Linear Quadratic Control
• Making Non-Stochastic Control (Almost) as Easy as Stochastic

Cited by in corpus (10)

• Non-stationary Online Learning with Memory and Non-stochastic Control
• Regret-optimal control in dynamic environments
• Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarking
• Dynamic Regret Minimization for Control of Non-stationary Linear Dynamical Systems
• Stable Online Control of Linear Time-Varying Systems
• Provable Regret Bounds for Deep Online Learning and Control
• Generating Adversarial Disturbances for Controller Verification
• Online estimation and control with optimal pathlength regret
• Adversarial Tracking Control via Strongly Adaptive Online Learning with Memory
• Improving Tractability of Real-Time Control Schemes via Simplified $\mathcal{S}$-Lemma