12 citations · 21 across the 20 of their papers we have counts for
4 papers · 1 filter
EControl: Fast Distributed Optimization with Compression and Error Control
Yuan Gao, Rustem Islamov, Sebastian Stich
Modern distributed training relies heavily on communication compression to reduce the communication overhead. In this work, we study algorithms employing a popular class of contrac…
AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms
Rustem Islamov, Mher Safaryan, Dan Alistarh
We analyze asynchronous-type algorithms for distributed SGD in the heterogeneous setting, where each worker has its own computation and communication speeds, as well as data distri…
Clip21: Error Feedback for Gradient Clipping
Sarit Khirirat, Eduard Gorbunov, Samuel Horváth +3
Motivated by the increasing popularity and importance of large-scale training under differential privacy (DP) constraints, we study distributed gradient methods with gradient clipp…
Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity
Konstantin Mishchenko, Rustem Islamov, Eduard Gorbunov +1
We present a partially personalized formulation of Federated Learning (FL) that strikes a balance between the flexibility of personalization and cooperativeness of global training.…