4 citations · 7 across the 5 of their papers we have counts for
7 papers
Better Batches, Better Privacy: Random Allocation for DP-BandMF
Ryan McKenna, Arun Ganesh
The current state of the art for batch selection and noise addition in differentially private training is a combination of DP-BandMF (informally, DP-SGD where the noise added in tw…
MAPLE: Metadata Augmented Private Language Evolution
Eli Chien, Yuzheng Hu, Ryan McKenna +3
Differentially private (DP) fine-tuning of large language models (LLMs) requires massive compute and full model access, which rules out state-of-the-art proprietary APIs for genera…
VaultGemma: A Differentially Private Gemma Model
Amer Sinha, Thomas Mesnard, Ryan McKenna +18
We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy. Pretrained on the identical data mixture used for the Gemm…
Correlated Noise Mechanisms for Differentially Private Learning
Krishna Pillutla, Jalaj Upadhyay, Christopher A. Choquette-Choo +9
This monograph explores the design and analysis of correlated noise mechanisms for differential privacy (DP), focusing on their application to private training of AI and machine le…
It's My Data Too: Private ML for Datasets with Multi-User Training Examples
Arun Ganesh, Ryan McKenna, Brendan McMahan +2
We initiate a study of algorithms for model training with user-level differential privacy (DP), where each example may be attributed to multiple users, which we call the multi-attr…
Scaling Laws for Differentially Private Language Models
Ryan McKenna, Yangsibo Huang, Amer Sinha +9
Scaling laws have emerged as important components of large language model (LLM) training as they can predict performance gains through scale, and provide guidance on important hype…
Fine-Tuning Large Language Models with User-Level Differential Privacy
Zachary Charles, Arun Ganesh, Ryan McKenna +4
We investigate practical and scalable algorithms for training large language models (LLMs) with user-level differential privacy (DP) in order to provably safeguard all the examples…