18 citations · 22 across the 2 of their papers we have counts for
5 papers · 1 filter
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari +5
Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically…
Federated Online and Bandit Convex Optimization
Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha +1
We study the problems of distributed online and bandit convex optimization against an adaptive adversary. We aim to minimize the average regret on machines working in parallel…
On the Effect of Defections in Federated Learning and How to Prevent Them
Minbiao Han, Kumar Kshitij Patel, Han Shao +1
Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model. There are sever…
Is Local SGD Better than Minibatch SGD?
Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich +5
We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method. Its theoretical foundations are c…
Communication trade-offs for synchronized distributed SGD with large step size
Kumar Kshitij Patel, Aymeric Dieuleveut
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds betw…