Submix: Practical Private Prediction for Large-Scale Language Models

Antonio Ginart, Laurens van der Maaten, James Zou +1

Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim. This is a vulnerability that can compromise the privacy of the model's…