873 citations · 5.4k across the 120 of their papers we have counts for
8 papers · 2 filters
A One-bit, Comparison-Based Gradient Estimator
HanQin Cai, Daniel Mckenzie, Wotao Yin +1
We study zeroth-order optimization for convex functions where we further assume that function evaluations are unavailable. Instead, one only has access to a $\textit{comparison ora…
An Improved Analysis of Stochastic Gradient Descent with Momentum
Yanli Liu, Yuan Gao, Wotao Yin
SGD with momentum (SGDM) has been widely applied in many machine learning tasks, and it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise manner. De…
Decentralized Learning with Lazy and Approximate Dual Gradients
Yanli Liu, Yuejiao Sun, Wotao Yin
This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neig…
Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
Tianyi Chen, Yuejiao Sun, Wotao Yin
Stochastic compositional optimization generalizes classic (non-compositional) stochastic optimization to the minimization of compositions of functions. Each composition may introdu…
A mean field game inverse problem
Lisang Ding, Wuchen Li, Stanley Osher +1
Mean-field games arise in various fields including economics, engineering, and machine learning. They study strategic decision making in large populations where the individuals int…
Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling
HanQin Cai, Daniel Mckenzie, Wotao Yin +1
We consider the problem of minimizing a high-dimensional objective function, which may include a regularization term, using (possibly noisy) evaluations of the function. Such optim…