5 papers
Gradient Methods with Online Scaling Part II. Practical Aspects
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This pap…
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
This paper establishes the theoretical foundations of the online scaled gradient methods (OSGM), a framework that utilizes online learning to adapt stepsizes and provably accelerat…
Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
This paper investigates the convergence properties of the hypergradient descent method (HDM), a 25-year-old heuristic originally proposed for adaptive stepsize selection in stochas…
When Does Primal Interior Point Method Beat Primal-dual in Linear Optimization?
Wenzhi Gao, Huikang Liu, Yinyu Ye +1
The primal-dual interior point method (IPM) is widely regarded as the most efficient IPM variant for linear optimization. In this paper, we demonstrate that the improved stability…
Gradient Methods with Online Scaling
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an onlin…