8 citations · 22 across the 25 of their papers we have counts for
25 papers
Stochastic Gradient Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Yinyu Ye +1
This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in arXiv:2505.23081 and arXiv…
Operator Splitting Methods with Online Scaling
Wanyu Zhang, Wenzhi Gao, Madeleine Udell
This paper develops a principled framework for automatically tuning preconditioners in operator splitting methods, including forward-backward splitting, Douglas-Rachford splitting,…
Tight Nonasymptotic Local Convergence of Sinkhorn-Knopp
Wenzhi Gao, Zhaonan Qu, Yinyu Ye +1
We revisit the Sinkhorn-Knopp (SK) algorithm for the matrix scaling problem. Despite extensive literature on the global convergence of SK and its variants, its local linear converg…
Small Gradient Norm Regret for Online Convex Optimization
Wenzhi Gao, Chang He, Madeleine Udell
This paper introduces a new problem-dependent regret measure for online convex optimization with smooth losses. The notion, which we call the regret, depends on the cumul…
A Smooth Approximation Framework for Weakly Convex Optimization
Qi Deng, Wenzhi Gao
Standard complexity analyses for weakly convex optimization rely on the Moreau envelope technique proposed by Davis and Drusvyatskiy (2019). The main insight is that nonsmooth algo…
New Results on the Polyak Stepsize: Tight Convergence Analysis and Universal Function Classes
Chang He, Wenzhi Gao, Bo Jiang +2
In this paper, we revisit a classical adaptive stepsize strategy for gradient descent: the Polyak stepsize (PolyakGD), originally proposed in Polyak (1969). We study the convergenc…
Gradient Methods with Online Scaling Part II. Practical Aspects
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This pap…
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
This paper establishes the theoretical foundations of the online scaled gradient methods (OSGM), a framework that utilizes online learning to adapt stepsizes and provably accelerat…
Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
Ya-Chi Chu, Wenzhi Gao, Yinyu Ye +1
This paper investigates the convergence properties of the hypergradient descent method (HDM), a 25-year-old heuristic originally proposed for adaptive stepsize selection in stochas…
Beyond Regret: Decoupling Learning and Decision-making in Online Linear Programming
Wenzhi Gao, Dongdong Ge, Chenyu Xue +2
Online linear programming plays an important role in both revenue management and resource allocation, and recent research has focused on developing efficient first-order online lea…
Wait-Less Offline Tuning and Re-solving for Online Decision Making
Jingruo Sun, Wenzhi Gao, Ellen Vitercik +1
Online linear programming (OLP) has found broad applications in revenue management and resource allocation. State-of-the-art OLP algorithms achieve low regret by repeatedly solving…
When Does Primal Interior Point Method Beat Primal-dual in Linear Optimization?
Wenzhi Gao, Huikang Liu, Yinyu Ye +1
The primal-dual interior point method (IPM) is widely regarded as the most efficient IPM variant for linear optimization. In this paper, we demonstrate that the improved stability…
Gradient Methods with Online Scaling
Wenzhi Gao, Ya-Chi Chu, Yinyu Ye +1
We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an onlin…
OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale
Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg +3
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than opt…
OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
Ali AhmadiTeshnizi, Wenzhi Gao, Madeleine Udell
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than opt…
Decoupling Learning and Decision-Making: Breaking the Barrier in Online Resource Allocation with First-Order Methods
Wenzhi Gao, Chunlin Sun, Chenyu Xue +2
Online linear programming plays an important role in both revenue management and resource allocation, and recent research has focused on developing efficient first-order online lea…
Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity
Wenzhi Gao, Qi Deng
This paper considers stochastic weakly convex optimization without the standard Lipschitz continuity assumption. Based on new adaptive regularization (stepsize) strategies, we show…
Scalable Approximate Optimal Diagonal Preconditioning
Wenzhi Gao, Zhaonan Qu, Madeleine Udell +1
We consider the problem of finding the optimal diagonal preconditioner for a positive definite matrix. Although this problem has been shown to be solvable and various methods have…
OptiMUS: Optimization Modeling Using MIP Solvers and large language models
Ali AhmadiTeshnizi, Wenzhi Gao, Madeleine Udell
Optimization problems are pervasive across various sectors, from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand ra…
Data-driven Mixed Integer Optimization through Probabilistic Multi-variable Branching
Yanguang Chen, Wenzhi Gao, Wanyu Zhang +3
In this paper, we propose a Pre-trained Mixed Integer Optimization framework (PreMIO) that accelerates online mixed integer program (MIP) solving with offline datasets and machine…
Delayed Stochastic Algorithms for Distributed Weakly Convex Optimization
Wenzhi Gao, Qi Deng
This paper studies delayed stochastic algorithms for weakly convex optimization in a distributed network with workers connected to a master node. Recently, Xu et al. 2022 showed th…
Optimal Diagonal Preconditioning
Zhaonan Qu, Wenzhi Gao, Oliver Hinder +2
Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although ther…
HDSDP: Software for Semidefinite Programming
Wenzhi Gao, Dongdong Ge, Yinyu Ye
HDSDP is a numerical software solving the semidefinite programming problems. The main framework of HDSDP resembles the dual-scaling interior point solver DSDP [BY2008] and several…
Solving Linear Programs with Fast Online Learning Algorithms
Wenzhi Gao, Dongdong Ge, Chunlin Sun +1
This paper presents fast first-order methods for solving linear programs (LPs) approximately. We adapt online linear programming algorithms to offline LPs and obtain algorithms tha…