1.7k citations · 1.9k across the 14 of their papers we have counts for
11 papers · 1 filter
Understanding Progressive Training Through the Framework of Randomized Coordinate Descent
Rafał Szlendak, Elnur Gasanov, Peter Richtárik
We propose a Randomized Progressive Training algorithm (RPT) -- a stochastic proxy for the well-known Progressive Training method (PT) (Karras et al., 2017). Originally designed to…
Improving Accelerated Federated Learning with Compression and Importance Sampling
Michał Grudzień, Grigory Malinovsky, Peter Richtárik
Federated Learning is a collaborative training framework that leverages heterogeneous data distributed across a vast number of clients. Since it is practically infeasible to reques…
Clip21: Error Feedback for Gradient Clipping
Sarit Khirirat, Eduard Gorbunov, Samuel Horváth +3
Motivated by the increasing popularity and importance of large-scale training under differential privacy (DP) constraints, we study distributed gradient methods with gradient clipp…
A Guide Through the Zoo of Biased SGD
Yury Demidovich, Grigory Malinovsky, Igor Sokolov +1
Stochastic Gradient Descent (SGD) is arguably the most important single algorithm in modern machine learning. Although SGD with unbiased gradient estimators has been studied extens…
Error Feedback Shines when Features are Rare
Peter Richtárik, Elnur Gasanov, Konstantin Burlachenko
We provide the first proof that gradient descent with greedy sparsification and error feedback $\left({\…
Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning
Kai Yi, Laurent Condat, Peter Richtárik
Federated Learning is an evolving machine learning paradigm, in which multiple clients perform computations based on their individual private data, interspersed by communication wi…