164 citations · 518 across the 32 of their papers we have counts for
3 papers · 1 filter
Training Production Language Models without Memorizing User Data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews +3
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (…
Understanding Unintended Memorization in Federated Learning
Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews +1
Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are o…
Training Keyword Spotting Models on Non-IID Data with Federated Learning
Andrew Hard, Kurt Partridge, Cameron Nguyen +5
We demonstrate that a production-quality keyword-spotting model can be trained on-device using federated learning and achieve comparable false accept and false reject rates to a ce…