30 citations · 65 across the 7 of their papers we have counts for
7 papers · 1 filter
An Improved Analysis of Stochastic Gradient Descent with Momentum
Yanli Liu, Yuan Gao, Wotao Yin
SGD with momentum (SGDM) has been widely applied in many machine learning tasks, and it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise manner. De…
Decentralized Learning with Lazy and Approximate Dual Gradients
Yanli Liu, Yuejiao Sun, Wotao Yin
This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neig…
Acceleration of SVRG and Katyusha X by Inexact Preconditioning
Yanli Liu, Fei Feng, Wotao Yin
Empirical risk minimization is an important class of optimization problems with many popular machine learning applications, and stochastic variance reduction methods are popular ch…
Acceleration of Primal-Dual Methods by Preconditioning and Simple Subproblem Procedures
Yanli Liu, Yunbei Xu, Wotao Yin
Primal-Dual Hybrid Gradient (PDHG) and Alternating Direction Method of Multipliers (ADMM) are two widely-used first-order optimization methods. They reduce a difficult problem to s…
Breaking the Span Assumption Yields Fast Finite-Sum Minimization
Robert Hannah, Yanli Liu, Daniel O'Connor +1
In this paper, we show that SVRG and SARAH can be modified to be fundamentally faster than all of the other standard algorithms that minimize the sum of smooth functions, such…
An Envelope for Davis-Yin Splitting and Strict Saddle Point Avoidance
Yanli Liu, Wotao Yin
It is known that operator splitting methods based on Forward Backward Splitting (FBS), Douglas-Rachford Splitting (DRS), and Davis-Yin Splitting (DYS) decompose a difficult optimiz…