30 citations · 87 across the 11 of their papers we have counts for
8 papers · 1 filter
An Improved Analysis of Stochastic Gradient Descent with Momentum
Yanli Liu, Yuan Gao, Wotao Yin
SGD with momentum (SGDM) has been widely applied in many machine learning tasks, and it is often applied with dynamic stepsizes and momentum weights tuned in a stagewise manner. De…
Decentralized Learning with Lazy and Approximate Dual Gradients
Yanli Liu, Yuejiao Sun, Wotao Yin
This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neig…
Acceleration of SVRG and Katyusha X by Inexact Preconditioning
Yanli Liu, Fei Feng, Wotao Yin
Empirical risk minimization is an important class of optimization problems with many popular machine learning applications, and stochastic variance reduction methods are popular ch…
Acceleration of Primal-Dual Methods by Preconditioning and Simple Subproblem Procedures
Yanli Liu, Yunbei Xu, Wotao Yin
Primal-Dual Hybrid Gradient (PDHG) and Alternating Direction Method of Multipliers (ADMM) are two widely-used first-order optimization methods. They reduce a difficult problem to s…
Breaking the Span Assumption Yields Fast Finite-Sum Minimization
Robert Hannah, Yanli Liu, Daniel O'Connor +1
In this paper, we show that SVRG and SARAH can be modified to be fundamentally faster than all of the other standard algorithms that minimize the sum of smooth functions, such…
An Envelope for Davis-Yin Splitting and Strict Saddle Point Avoidance
Yanli Liu, Wotao Yin
It is known that operator splitting methods based on Forward Backward Splitting (FBS), Douglas-Rachford Splitting (DRS), and Davis-Yin Splitting (DYS) decompose a difficult optimiz…
Douglas--Rachford Splitting and ADMM for Pathological Convex Optimization
Ernest K. Ryu, Yanli Liu, Wotao Yin
Despite the vast literature on DRS and ADMM, there has been very little work analyzing their behavior under pathologies. Most analyses assume a primal solution exists, a dual solut…
A New Use of Douglas-Rachford Splitting and ADMM for Identifying Infeasible, Unbounded, and Pathological Conic Programs
Yanli Liu, Ernest K. Ryu, Wotao Yin
In this paper, we present a method for identifying infeasible, unbounded, and pathological conic programs based on Douglas-Rachford splitting, or equivalently ADMM. When an optimiz…