66 citations · 324 across the 32 of their papers we have counts for
6 papers · 2 filters
Faster Rates for Compressed Federated Learning with Client-Variance Reduction
Haoyu Zhao, Konstantin Burlachenko, Zhize Li +1
Due to the communication bottleneck in distributed and federated learning applications, algorithms using communication compression have attracted significant attention and are wide…
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
Ilyas Fatkhullin, Igor Sokolov, Eduard Gorbunov +2
First proposed by Seide (2014) as a heuristic, error feedback (EF) is a very popular mechanism for enforcing convergence of distributed gradient-based optimization methods enhanced…
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning
Haoyu Zhao, Zhize Li, Peter Richtárik
Federated Averaging (FedAvg, also known as Local-SGD) (McMahan et al., 2017) is a classical federated learning algorithm in which clients run multiple local SGD steps before commun…
CANITA: Faster Rates for Distributed Convex Optimization with Communication Compression
Zhize Li, Peter Richtárik
Due to the high communication cost in distributed and federated learning, methods relying on compressed communication are becoming increasingly popular. Besides, the best theoretic…
ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computation
Zhize Li, Slavomír Hanzely, Peter Richtárik
We propose ZeroSARAH -- a novel variant of the variance-reduced method SARAH (Nguyen et al., 2017) -- for minimizing the average of a large number of nonconvex functions $\frac{1}{…
MARINA: Faster Non-Convex Distributed Learning with Compression
Eduard Gorbunov, Konstantin Burlachenko, Zhize Li +1
We develop and analyze MARINA: a new communication efficient method for non-convex distributed learning over heterogeneous datasets. MARINA employs a novel communication compressio…