164 citations · 341 across the 9 of their papers we have counts for
9 papers
UserLibri: A Dataset for ASR Personalization Using Only Text
Theresa Breiner, Swaroop Ramaswamy, Ehsan Variani +6
Personalization of speech models on mobile devices (on-device personalization) is an active area of research, but more often than not, mobile devices have more text-only data than…
Public Data-Assisted Mirror Descent for Private Model Training
Ehsan Amid, Arun Ganesh, Rajiv Mathews +6
In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training. (Here, public d…
Revealing and Protecting Labels in Distributed Training
Trung Dang, Om Thakkar, Swaroop Ramaswamy +3
Distributed learning paradigms such as federated learning often involve transmission of model updates, or gradients, over a network, thereby avoiding transmission of private data.…
A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter It
Trung Dang, Om Thakkar, Swaroop Ramaswamy +3
End-to-end Automatic Speech Recognition (ASR) models are commonly trained over spoken utterances using optimization methods like Stochastic Gradient Descent (SGD). In distributed s…
Training Production Language Models without Memorizing User Data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews +3
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (…
Understanding Unintended Memorization in Federated Learning
Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews +1
Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are o…
Generative Models for Effective ML on Private, Decentralized Datasets
Sean Augenstein, H. Brendan McMahan, Daniel Ramage +5
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact. Manual inspection of ra…
Federated Learning for Emoji Prediction in a Mobile Keyboard
Swaroop Ramaswamy, Rajiv Mathews, Kanishka Rao +1
We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard. We demonstrate the usefulness of transfer learning for predicting emoji b…
Differentially Private Learning with Adaptive Clipping
Galen Andrew, Om Thakkar, H. Brendan McMahan +1
Existing approaches for training neural networks with user-level differential privacy (e.g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contri…