7 papers
Behavioral Canaries: Auditing Private Retrieved Context Usage in RL Fine-Tuning
Chaoran Chen, Dayu Yuan, Peter Kairouz
In agentic workflows, LLMs frequently process retrieved contexts that are legally protected from further training. However, auditors currently lack a reliable way to verify if a pr…
Can Large Language Models Really Recognize Your Name?
Dzung Pham, Peter Kairouz, Niloofar Mireshghallah +3
Large language models (LLMs) are increasingly being used in privacy pipelines to detect and remedy sensitive data leakage. These solutions often rely on the premise that LLMs can r…
MAPLE: Metadata Augmented Private Language Evolution
Eli Chien, Yuzheng Hu, Ryan McKenna +3
Differentially private (DP) fine-tuning of large language models (LLMs) requires massive compute and full model access, which rules out state-of-the-art proprietary APIs for genera…
Redirection for Erasing Memory (REM): Towards a universal unlearning method for corrupted data
Stefan Schoepf, Michael Curtis Mozer, Nicole Elyse Mitchell +4
Machine unlearning is studied for a multitude of tasks, but specialization of unlearning methods to particular tasks has made their systematic comparison challenging. To address th…
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research
A. Feder Cooper, Christopher A. Choquette-Choo, Miranda Bogen +34
"Machine unlearning" is a popular proposed solution for mitigating the existence of content in an AI model that is problematic for legal or moral reasons, including privacy, copyri…
Language Models May Verbatim Complete Text They Were Not Explicitly Trained On
Ken Ziyu Liu, Christopher A. Choquette-Choo, Matthew Jagielski +4
An important question today is whether a given text was used to train a large language model (LLM). A \emph{completion} test is often employed: check if the LLM completes a suffici…