Certainty Equivalence is Efficient for Linear Quadratic Control
arXiv:1902.07826
Abstract
We study the performance of the certainty equivalent controller on Linear Quadratic (LQ) control problems with unknown transition dynamics. We show that for both the fully and partially observed settings, the sub-optimality gap between the cost incurred by playing the certainty equivalent controller on the true system and the cost incurred by using the optimal LQ controller enjoys a fast statistical rate, scaling as the square of the parameter error. To the best of our knowledge, our result is the first sub-optimality guarantee in the partially observed Linear Quadratic Gaussian (LQG) setting. Furthermore, in the fully observed Linear Quadratic Regulator (LQR), our result improves upon recent work by Dean et al. (2017), who present an algorithm achieving a sub-optimality gap linear in the parameter error. A key part of our analysis relies on perturbation bounds for discrete Riccati equations. We provide two new perturbation bounds, one that expands on an existing result from Konstantinov et al. (1993), and another based on a new elementary proof strategy.
In the current version we extended our analysis to the case of partially observable systems, i.e. we provided a suboptimality analysis for the Linear Quadratic Gaussian (LQG) setting
References in corpus (5)
- Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator
- Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems
- Near optimal finite time identification of arbitrary linear dynamical systems
- Non-Asymptotic Analysis of Robust Control from Coarse-Grained Identification
- Model-Free Linear Quadratic Control via Reduction to Expert Prediction
Cited by in corpus (6)
- Data-driven control via Petersen's lemma
- Meta-Adaptive Nonlinear Control: Theory and Algorithms
- Near-Optimal Design of Safe Output Feedback Controllers from Noisy Data
- Sample Complexity of Data-Driven Stochastic LQR with Multiplicative Uncertainty
- Linear quadratic control of nonlinear systems with Koopman operator learning and the Nyström method
- Ensuring the Safety of Uncertified Linear State-Feedback Controllers via Switching