66 citations · 324 across the 32 of their papers we have counts for
9 papers · 1 filter
Faster Rates for Compressed Federated Learning with Client-Variance Reduction
Haoyu Zhao, Konstantin Burlachenko, Zhize Li +1
Due to the communication bottleneck in distributed and federated learning applications, algorithms using communication compression have attracted significant attention and are wide…
Factored couplings in multi-marginal optimal transport via difference of convex programming
Quang Huy Tran, Hicham Janati, Ievgen Redko +2
Optimal transport (OT) theory underlies many emerging machine learning (ML) methods nowadays solving a wide range of tasks such as generative modeling, transfer learning and inform…
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
Ilyas Fatkhullin, Igor Sokolov, Eduard Gorbunov +2
First proposed by Seide (2014) as a heuristic, error feedback (EF) is a very popular mechanism for enforcing convergence of distributed gradient-based optimization methods enhanced…
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning
Haoyu Zhao, Zhize Li, Peter Richtárik
Federated Averaging (FedAvg, also known as Local-SGD) (McMahan et al., 2017) is a classical federated learning algorithm in which clients run multiple local SGD steps before commun…
CANITA: Faster Rates for Distributed Convex Optimization with Communication Compression
Zhize Li, Peter Richtárik
Due to the high communication cost in distributed and federated learning, methods relying on compressed communication are becoming increasingly popular. Besides, the best theoretic…
A Short Note of PAGE: Optimal Convergence Rates for Nonconvex Optimization
Zhize Li
In this note, we first recall the nonconvex problem setting and introduce the optimal PAGE algorithm (Li et al., ICML'21). Then we provide a simple and clean convergence analysis o…