86 citations · 201 across the 19 of their papers we have counts for
4 papers · 1 filter
Training Production Language Models without Memorizing User Data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews +3
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (…
Privacy Amplification via Random Check-Ins
Borja Balle, Peter Kairouz, H. Brendan McMahan +2
Differentially Private Stochastic Gradient Descent (DP-SGD) forms a fundamental building block in many applications for learning over sensitive data. Two standard approaches, priva…
Understanding Unintended Memorization in Federated Learning
Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews +1
Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are o…
Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent
Shuang Song, Thomas Steinke, Om Thakkar +1
We revisit the well-studied problem of differentially private empirical risk minimization (ERM). We show that for unconstrained convex generalized linear models (GLMs), one can obt…