3k citations · 9k across the 154 of their papers we have counts for
14 papers · 2 filters
A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!
Dmitry Kovalev, Anastasia Koloskova, Martin Jaggi +2
Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. In many scenarios communication between devices is energy dem…
Linearly Converging Error Compensated SGD
Eduard Gorbunov, Dmitry Kovalev, Dmitry Makarenko +1
In this paper, we propose a unified analysis of variants of distributed SGD with arbitrary compressions and delayed updates. Our framework is general enough to cover different vari…
Distributed Proximal Splitting Algorithms with Rates and Acceleration
Laurent Condat, Grigory Malinovsky, Peter Richtárik
We analyze several generic proximal splitting algorithms well suited for large-scale convex nonsmooth optimization. We derive sublinear and linear convergence results with new rate…
Error Compensated Distributed SGD Can Be Accelerated
Xun Qian, Peter Richtárik, Tong Zhang
Gradient compression is a recent and increasingly popular technique for reducing the communication cost in distributed training of large-scale machine learning models. In this work…
A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimization
Zhize Li, Peter Richtárik
In this paper, we study the performance of a large family of SGD variants in the smooth nonconvex regime. To this end, we propose a generic and flexible assumption capable of accur…
Optimal and Practical Algorithms for Smooth and Strongly Convex Decentralized Optimization
Dmitry Kovalev, Adil Salim, Peter Richtárik
We consider the task of decentralized minimization of the sum of smooth strongly convex functions stored across the nodes of a network. For this problem, lower bounds on the number…