Online Optimal Control with Affine Constraints
arXiv:2010.04891
Abstract
This paper considers online optimal control with affine constraints on the states and actions under linear dynamics with bounded random disturbances. The system dynamics and constraints are assumed to be known and time-invariant but the convex stage cost functions change adversarially. To solve this problem, we propose Online Gradient Descent with Buffer Zones (OGD-BZ). Theoretically, we show that OGD-BZ with proper parameters can guarantee the system to satisfy all the constraints despite any admissible disturbances. Further, we investigate the policy regret of OGD-BZ, which compares OGD-BZ's performance with the performance of the optimal linear policy in hindsight. We show that OGD-BZ can achieve a policy regret upper bound that is the square root of the horizon length multiplied by some logarithmic terms of the horizon length under proper algorithm parameters.
arXiv commentAccepted by AAAI 2021
References in corpus (11)
- Safe Reinforcement Learning Using Robust MPC
- Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator
- Online convex optimization for cumulative constraints
- Online Optimal Control with Linear Dynamics and Predictions: Algorithms and Regret Analysis
- Online Control with Adversarial Disturbances
- Logarithmic Regret for Online Control
- Logarithmic Regret for Adversarial Online Control
- An online convex optimization algorithm for controlling linear systems with state and input constraints
- Linear Controller Design for Chance Constrained Systems
- Regret-optimal control in dynamic environments
- Leveraging Predictions in Smoothed Online Convex Optimization via Gradient-based Algorithms