8 citations · 22 across the 25 of their papers we have counts for
16 papers · 1 filter
Stochastic Gradient Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Yinyu Ye +1
This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in arXiv:2505.23081 and arXiv…
Operator Splitting Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Madeleine Udell
This paper develops a principled framework for automatically tuning preconditioners in operator splitting methods, including forward-backward splitting, Douglas-Rachford splitting,…
Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp
Wenzhi Gao, Zhaonan Qu, Yinyu Ye +1
We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear converg…
A Smooth Approximation Framework for Weakly Convex Optimization
Qi Deng, Wenzhi Gao
Standard complexity analyses for weakly convex optimization rely on the Moreau envelope technique proposed by Davis and Drusvyatskiy (2019). The main insight is that nonsmooth algo…
New Results on the Polyak Stepsize: Tight Convergence Analysis and Universal Function Classes
Chang He, Wenzhi Gao, Bo Jiang +2
In this paper, we revisit a classical adaptive stepsize strategy for gradient descent: the Polyak stepsize (PolyakGD), originally proposed in Polyak (1969). We study the convergenc…
Gradient Methods with Online Scaling Part II. Practical Aspects
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This pap…
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
This paper establishes the theoretical foundations of the online scaled gradient methods (OSGM), a framework that utilizes online learning to adapt stepsizes and provably accelerat…
Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
This paper investigates the convergence properties of the hypergradient descent method (HDM), a 25-year-old heuristic originally proposed for adaptive stepsize selection in stochas…
When Does Primal Interior Point Method Beat Primal-dual in Linear Optimization?
Wenzhi Gao, Huikang Liu, Yinyu Ye +1
The primal-dual interior point method (IPM) is widely regarded as the most efficient IPM variant for linear optimization. In this paper, we demonstrate that the improved stability…
Gradient Methods with Online Scaling
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an onlin…
Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity
Wenzhi Gao, Qi Deng
This paper considers stochastic weakly convex optimization without the standard Lipschitz continuity assumption. Based on new adaptive regularization (stepsize) strategies, we show…
Data-driven Mixed Integer Optimization through Probabilistic Multi-variable Branching
Yanguang Chen, Wenzhi Gao, Wanyu Zhang +3
In this paper, we propose a Pre-trained Mixed Integer Optimization framework (PreMIO) that accelerates online mixed integer program (MIP) solving with offline datasets and machine…
Delayed Stochastic Algorithms for Distributed Weakly Convex Optimization
Wenzhi Gao, Qi Deng
This paper studies delayed stochastic algorithms for weakly convex optimization in a distributed network with workers connected to a master node. Recently, Xu et al. 2022 showed th…
Optimal Diagonal Preconditioning
Zhaonan Qu, Wenzhi Gao, Oliver Hinder +2
Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although ther…
Solving Linear Programs with Fast Online Learning Algorithms
Wenzhi Gao, Dongdong Ge, Chunlin Sun +1
This paper presents fast first-order methods for solving linear programs (LPs) approximately. We adapt online linear programming algorithms to offline LPs and obtain algorithms tha…
Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimization
Qi Deng, Wenzhi Gao
Stochastic model-based methods have received increasing attention lately due to their appealing robustness to the stepsize selection and provable efficiency guarantee. We make two…