10 citations · 78 across the 72 of their papers we have counts for
6 papers · 2 filters
Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates
Ahmad Rammal, Kaja Gruntkowska, Nikita Fedin +2
Byzantine robustness is an essential feature of algorithms for certain distributed optimization problems, typically encountered in collaborative/federated learning. These problems…
Variance Reduced Distributed Non-Convex Optimization Using Matrix Stepsizes
Hanmin Li, Avetik Karagulyan, Peter Richtárik
Matrix-stepsized gradient descent algorithms have been shown to have superior performance in non-convex optimization problems compared to their scalar counterparts. The det-CGD alg…
High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
Eduard Gorbunov, Abdurakhmon Sadiev, Marina Danilova +5
High-probability analysis of stochastic first-order optimization methods under mild assumptions on the noise has been gaining a lot of attention in recent years. Typically, gradien…
Error Feedback Shines when Features are Rare
Peter Richtárik, Elnur Gasanov, Konstantin Burlachenko
We provide the first proof that gradient descent with greedy sparsification and error feedback $\left({\…
2Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
Alexander Tyurin, Peter Richtárik
We consider distributed convex optimization problems in the regime when the communication between the server and the workers is expensive in both uplink and downlink directions. We…
Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity
Xun Qian, Hanze Dong, Tong Zhang +1
Communication overhead is well known to be a key bottleneck in large scale distributed learning, and a particularly successful class of methods which help to overcome this bottlene…