Learning the Globally Optimal Distributed LQ Regulator

Abstract

We study model-free learning methods for the output-feedback Linear Quadratic (LQ) control problem in finite-horizon subject to subspace constraints on the control policy. Subspace constraints naturally arise in the field of distributed control and present a significant challenge in the sense that standard model-based optimization and learning leads to intractable numerical programs in general. Building upon recent results in zeroth-order optimization, we establish model-free sample-complexity bounds for the class of distributed LQ problems where a local gradient dominance constant exists on any sublevel set of the cost function. %which admit a local gradient dominance constant valid on the sublevel set of the cost function. We prove that a fundamental class of distributed control problems - commonly referred to as Quadratically Invariant (QI) problems - as well as others possess this property. To the best of our knowledge, our result is the first sample-complexity bound guarantee on learning globally optimal distributed output-feedback control policies.

Soon to appear in Proceedings of Machine Learning Research, Vol. 120. Presented at L4DC 2020

References in corpus (4)

• Stochastic Recursive Gradient Algorithm for Nonconvex Optimization
• Sparsity Invariance for Convex Design of Distributed Controllers
• First Order Methods For Globally Optimal Distributed Controllers Beyond Quadratic Invariance
• Efficient Learning of Distributed Linear-Quadratic Controllers

Cited by in corpus (14)

• System-level, Input-output and New Parameterizations of Stabilizing Controllers, and Their Numerical Computation
• Near-Optimal Design of Safe Output Feedback Controllers from Noisy Data
• Asynchronous Distributed Reinforcement Learning for LQR Control via Zeroth-Order Block Coordinate Descent
• Optimization Landscape of Gradient Descent for Discrete-time Static Output Feedback
• Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRs
• Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexity
• Primal-dual Learning for the Model-free Risk-constrained Linear Quadratic Regulator
• Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation
• Regret Analysis of Distributed Online LQR Control for Unknown LTI Systems
• Learning Partially Observed Linear Dynamical Systems from Logarithmic Number of Samples
• On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structure
• Sample Complexity of the Robust LQG Regulator with Coprime Factors Uncertainty
• Model-Free $\mu$ Synthesis via Adversarial Reinforcement Learning
• Optimal Decentralized Control for Uncertain Systems by Symmetric Gauss-Seidel Semi-Proximal ALM