88 citations · 172 across the 10 of their papers we have counts for
10 papers
CADA: Communication-Adaptive Distributed Adam
Tianyi Chen, Ziye Guo, Yuejiao Sun +1
Stochastic gradient descent (SGD) has taken the stage as the primary workhorse for large-scale machine learning. It is often used with its adaptive variants such as AdaGrad, Adam,…
Decentralized Learning with Lazy and Approximate Dual Gradients
Yanli Liu, Yuejiao Sun, Wotao Yin
This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neig…
Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
Tianyi Chen, Yuejiao Sun, Wotao Yin
Stochastic compositional optimization generalizes classic (non-compositional) stochastic optimization to the minimization of compositions of functions. Each composition may introdu…
VAFL: a Method of Vertical Asynchronous Federated Learning
Tianyi Chen, Xiao Jin, Yuejiao Sun +1
Horizontal Federated learning (FL) handles multi-client data that share the same set of features, and vertical FL trains a better predictor that combine all the features from diffe…
LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning
Tianyi Chen, Yuejiao Sun, Wotao Yin
This paper targets solving distributed machine learning problems such as federated learning in a communication-efficient fashion. A class of new stochastic gradient descent (SGD) a…
General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme
Tao Sun, Yuejiao Sun, Dongsheng Li +1
The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective funct…
Decentralized Markov Chain Gradient Descent
Tao Sun, Dongsheng Li
Decentralized stochastic gradient method emerges as a promising solution for solving large-scale machine learning problems. This paper studies the decentralized Markov chain gradie…
Markov Chain Block Coordinate Descent
Tao Sun, Yuejiao Sun, Yangyang Xu +1
The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a serie…
On Markov Chain Gradient Descent
Tao Sun, Yuejiao Sun, Wotao Yin
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and enginee…
Run-and-Inspect Method for Nonconvex Optimization and Global Optimality Bounds for R-Local Minimizers
Yifan Chen, Yuejiao Sun, Wotao Yin
Many optimization algorithms converge to stationary points. When the underlying problem is nonconvex, they may get trapped at local minimizers and occasionally stagnate near saddle…