8 citations · 18 across the 16 of their papers we have counts for
16 papers
Stochastic Gradient Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Yinyu Ye +1
This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in arXiv:2505.23081 and arXiv…
Operator Splitting Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Madeleine Udell
This paper develops a principled framework for automatically tuning preconditioners in operator splitting methods, including forward-backward splitting, Douglas-Rachford splitting,…
Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp
Wenzhi Gao, Zhaonan Qu, Yinyu Ye +1
We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear converg…
Small Gradient Norm Regret for Online Convex Optimization
Wenzhi Gao, Chang He, Madeleine Udell
This paper introduces a new problem-dependent regret measure for online convex optimization with smooth losses. The notion, which we call the regret, depends on the cumul…
SnareNet: Flexible Repair Layers for Neural Networks with Hard Constraints
Ya-Chi Chu, Alkiviades Boukas, Madeleine Udell
Neural networks are increasingly used as fast surrogate models across various domains, but unconstrained predictions can violate physical, operational, or safety requirements. We p…
New Results on the Polyak Stepsize: Tight Convergence Analysis and Universal Function Classes
Chang He, Wenzhi Gao, Bo Jiang +2
In this paper, we revisit a classical adaptive stepsize strategy for gradient descent: the Polyak stepsize (PolyakGD), originally proposed in Polyak (1969). We study the convergenc…
Gradient Methods with Online Scaling Part II. Practical Aspects
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This pap…
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
This paper establishes the theoretical foundations of the online scaled gradient methods (OSGM), a framework that utilizes online learning to adapt stepsizes and provably accelerat…
Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
This paper investigates the convergence properties of the hypergradient descent method (HDM), a 25-year-old heuristic originally proposed for adaptive stepsize selection in stochas…
When Does Primal Interior Point Method Beat Primal-dual in Linear Optimization?
Wenzhi Gao, Huikang Liu, Yinyu Ye +1
The primal-dual interior point method (IPM) is widely regarded as the most efficient IPM variant for linear optimization. In this paper, we demonstrate that the improved stability…
Gradient Methods with Online Scaling
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an onlin…
OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale
Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg +3
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than opt…
Randomized Nyström Preconditioned Interior Point-Proximal Method of Multipliers
Ya-Chi Chu, Luiz-Rafael Santos, Madeleine Udell
We present a new algorithm for convex separable quadratic programming (QP) called Nys-IP-PMM, a regularized interior-point solver that uses low-rank structure to accelerate solutio…
OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
Ali AhmadiTeshnizi, Wenzhi Gao, Madeleine Udell
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than opt…
Scalable Approximate Optimal Diagonal Preconditioning
Wenzhi Gao, Zhaonan Qu, Madeleine Udell +1
We consider the problem of finding the optimal diagonal preconditioner for a positive definite matrix. Although this problem has been shown to be solvable and various methods have…
OptiMUS: Optimization Modeling Using MIP Solvers and large language models
Ali AhmadiTeshnizi, Wenzhi Gao, Madeleine Udell
Optimization problems are pervasive across various sectors, from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand ra…