On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator

Abstract

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a unique saddle point, which simultaneously recovers the globally optimal policy and reward function. We hope our results may serve as a small step towards understanding and taming the instability in imitation learning as well as in more general non-convex-concave alternating minimax optimization that arises from reinforcement learning and generative adversarial learning.

References in corpus (2)

• Learning One-hidden-layer Neural Networks with Landscape Design
• Nonconvex Statistical Optimization: Minimax-Optimal Sparse PCA in Polynomial Time

Cited by in corpus (5)

• Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems
• On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost
• On Computation and Generalization of Generative Adversarial Imitation Learning
• When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence
• Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation