Hidden Star-Convexity in Policy Optimization for Gain-Scheduled LQR: Extended Version

Abstract

We study policy optimization for gain-scheduled linear quadratic regulation, where one schedule of gains, interpolated through fixed weighting functions, is optimized against a family of plants. The resulting cost can develop spurious local minima, and existing convergence certificates are either local or severely conservative. We establish an exact identity: when the gradient of the cost is evaluated with the minimizer's closed-loop covariances, the scheduled cost is star-convex about the minimizer. The identity holds on the entire feasible set, for any parametrization of the schedule. Convergence is governed by a single dimensionless ratio. Wherever the ratio satisfies a threshold condition, gradient descent converges linearly to the optimum on entire sublevel regions at an explicit rate; at every spurious stationary point the condition necessarily fails. Experiments that maximize the ratio directly show the threshold to be an active boundary of the landscape. This extended version contains the complete proofs and additional numerical studies omitted from the letter for space.

16 pages, 3 figures. Extended version of a manuscript submitted to IEEE Control Systems Letters. Code: https://github.com/Rainlabuw/hidden-star-convexity