45 citations · 113 across the 7 of their papers we have counts for
7 papers
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari +5
Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically…
Federated Online and Bandit Convex Optimization
Kumar Kshitij Patel, Lingxiao Wang, Aadirupa Saha +1
We study the problems of distributed online and bandit convex optimization against an adaptive adversary. We aim to minimize the average regret on machines working in parallel…
On the Effect of Defections in Federated Learning and How to Prevent Them
Minbiao Han, Kumar Kshitij Patel, Han Shao +1
Federated learning is a machine learning protocol that enables a large population of agents to collaborate over multiple rounds to produce a single consensus model. There are sever…
A Stochastic Newton Algorithm for Distributed Convex Optimization
Brian Bullins, Kumar Kshitij Patel, Ohad Shamir +2
We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same p…
Minibatch vs Local SGD for Heterogeneous Distributed Learning
Blake Woodworth, Kumar Kshitij Patel, Nathan Srebro
We analyze Local SGD (aka parallel or federated SGD) and Minibatch SGD in the heterogeneous distributed setting, where each machine has access to stochastic gradient estimates for…
Is Local SGD Better than Minibatch SGD?
Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich +5
We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method. Its theoretical foundations are c…
Communication trade-offs for synchronized distributed SGD with large step size
Kumar Kshitij Patel, Aymeric Dieuleveut
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds betw…