71 citations · 116 across the 15 of their papers we have counts for
16 papers
Learning from Synthetic Data: Limitations of ERM
Kareem Amin, Alex Bie, Weiwei Kong +2
The prevalence and low cost of LLMs have led to a rise of synthetic content. From review sites to court documents, "natural" content has been contaminated by data points that appea…
Clustering and Median Aggregation Improve Differentially Private Inference
Kareem Amin, Salman Avestimehr, Sara Babakniya +4
Differentially private (DP) language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large langua…
Escaping Collapse: The Strength of Weak Data for Large Language Model Training
Kareem Amin, Sara Babakniya, Alex Bie +3
Synthetically-generated data plays an increasingly larger role in training large language models. However, while synthetic data has been found to be useful, studies have also shown…
Private prediction for large-scale synthetic text generation
Kareem Amin, Alex Bie, Weiwei Kong +5
We present an approach for generating differentially private synthetic text using large language models (LLMs), via private prediction. In the private prediction framework, we only…
Lexicographic Optimization: Algorithms and Stability
Jacob Abernethy, Robert E. Schapire, Umar Syed
A lexicographic maximum of a set is a vector in whose smallest component is as large as possible, and subject to that requirement, whose second small…
Harnessing large-language models to generate private synthetic text
Alexey Kurakin, Natalia Ponomareva, Umar Syed +2
Differentially private training algorithms like DP-SGD protect sensitive training data by ensuring that trained models do not reveal private information. An alternative approach, w…