10 papers · 1 filter
Stochastic Gradient Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Yinyu Ye +1
This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in arXiv:2505.23081 and arXiv…
Operator Splitting Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Madeleine Udell
This paper develops a principled framework for automatically tuning preconditioners in operator splitting methods, including forward-backward splitting, Douglas-Rachford splitting,…
Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp
Wenzhi Gao, Zhaonan Qu, Yinyu Ye +1
We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear converg…
New Results on the Polyak Stepsize: Tight Convergence Analysis and Universal Function Classes
Chang He, Wenzhi Gao, Bo Jiang +2
In this paper, we revisit a classical adaptive stepsize strategy for gradient descent: the Polyak stepsize (PolyakGD), originally proposed in Polyak (1969). We study the convergenc…
Gradient Methods with Online Scaling Part II. Practical Aspects
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This pap…
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
This paper establishes the theoretical foundations of the online scaled gradient methods (OSGM), a framework that utilizes online learning to adapt stepsizes and provably accelerat…