1.6k citations · 3.6k across the 48 of their papers we have counts for
38 papers · 1 filter
Faster Rates For Federated Variational Inequalities
Guanghui Wang, Satyen Kale
In this paper, we study federated optimization for solving stochastic variational inequalities (VIs), a problem that has attracted growing attention in recent years. Despite substa…
Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
Ahmed Khaled, Satyen Kale, Arthur Douillard +3
Modern machine learning often requires training with large batch size, distributed data, and massively parallel compute hardware (like mobile and other edge devices or distributed…
Asynchronous Local-SGD Training for Language Modeling
Bo Liu, Rachita Chhaparia, Arthur Douillard +5
Local stochastic gradient descent (Local-SGD), also referred to as federated averaging, is an approach to distributed optimization where each device performs more than one SGD upda…
On the Convergence of Federated Averaging with Cyclic Client Participation
Yae Jee Cho, Pranay Sharma, Gauri Joshi +3
Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL). Previous convergence analyses of FedAvg either assume full cl…
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent
Satyen Kale, Jason D. Lee, Chris De Sa +2
Stochastic Gradient Descent (SGD) has been the method of choice for learning large-scale non-convex models. While a general analysis of when SGD works has been elusive, there has b…
Self-Consistency of the Fokker-Planck Equation
Zebang Shen, Zhenfu Wang, Satyen Kale +3
The Fokker-Planck equation (FPE) is the partial differential equation that governs the density evolution of the Itô process and is of great importance to the literature of statisti…