6.3k citations · 20.9k across the 54 of their papers we have counts for
4 papers · 1 filter
Training Production Language Models without Memorizing User Data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews +3
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (…
Privacy Amplification via Random Check-Ins
Borja Balle, Peter Kairouz, H. Brendan McMahan +2
Differentially Private Stochastic Gradient Descent (DP-SGD) forms a fundamental building block in many applications for learning over sensitive data. Two standard approaches, priva…
Adaptive Federated Optimization
Sashank Reddi, Zachary Charles, Manzil Zaheer +5
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own trainin…
Is Local SGD Better than Minibatch SGD?
Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich +5
We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method. Its theoretical foundations are c…