6 papers · 1 filter
ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?
Peihan Liu, Lucas Rosenblatt, Weiwei Kong +7
Differentially private (DP) text synthesis promises to unlock sensitive corpora for model training, but it remains unclear whether DP synthetic data transmits genuinely new knowled…
JAX-Privacy: A library for differentially private machine learning
Ryan McKenna, Galen Andrew, Borja Balle +6
JAX-Privacy is a library designed to simplify the deployment of robust and performant mechanisms for differentially private machine learning. Guided by design principles of usabili…
Learning from Synthetic Data: Limitations of ERM
Kareem Amin, Alex Bie, Weiwei Kong +2
The prevalence and low cost of LLMs have led to a rise of synthetic content. From review sites to court documents, "natural" content has been contaminated by data points that appea…
Escaping Collapse: The Strength of Weak Data for Large Language Model Training
Kareem Amin, Sara Babakniya, Alex Bie +3
Synthetically-generated data plays an increasingly larger role in training large language models. However, while synthetic data has been found to be useful, studies have also shown…
Clustering and Median Aggregation Improve Differentially Private Inference
Kareem Amin, Salman Avestimehr, Sara Babakniya +4
Differentially private (DP) language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large langua…
Private prediction for large-scale synthetic text generation
Kareem Amin, Alex Bie, Weiwei Kong +5
We present an approach for generating differentially private synthetic text using large language models (LLMs), via private prediction. In the private prediction framework, we only…