5 papers · 1 filter
Gradient Methods with Online Scaling Part II. Practical Aspects
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This pap…
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
This paper establishes the theoretical foundations of the online scaled gradient methods (OSGM), a framework that utilizes online learning to adapt stepsizes and provably accelerat…
Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
This paper investigates the convergence properties of the hypergradient descent method (HDM), a 25-year-old heuristic originally proposed for adaptive stepsize selection in stochas…
Randomized Nyström Preconditioned Interior Point-Proximal Method of Multipliers
Ya-Chi Chu, Luiz-Rafael Santos, Madeleine Udell
We present a new algorithm for convex separable quadratic programming (QP) called Nys-IP-PMM, a regularized interior-point solver that uses low-rank structure to accelerate solutio…
Gradient Methods with Online Scaling
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an onlin…