49 citations · 69 across the 27 of their papers we have counts for
12 papers · 1 filter
Extractable Memorization From First Principles
A. Feder Cooper, Marika Swanberg, Jamie Hayes +6
Recent work on extractable memorization in language models suffers from two contrasting validity problems. Some studies overstate extraction, for example, by using sequences too sh…
The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections
Milad Nasr, Nicholas Carlini, Chawin Sitawarin +11
How should we evaluate the robustness of language model defenses? Current defenses against jailbreaks and prompt injections (which aim to prevent an attacker from eliciting harmful…
Cascading Adversarial Bias from Injection to Distillation in Language Models
Harsh Chaudhari, Jamie Hayes, Matthew Jagielski +3
Model distillation has become essential for creating smaller, deployable language models that retain larger system capabilities. However, widespread deployment raises concerns abou…
Large Language Models Can Verbatim Reproduce Long Malicious Sequences
Sharon Lin, Krishnamurthy, Dvijotham +4
Backdoor attacks on machine learning models have been extensively studied, primarily within the computer vision domain. Originally, these attacks manipulated classifiers to generat…
Interpreting the Repeated Token Phenomenon in Large Language Models
Itay Yona, Ilia Shumailov, Jamie Hayes +2
Large Language Models (LLMs), despite their impressive capabilities, often fail to accurately repeat a single word when prompted to, and instead output unrelated text. This unexpla…
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research
A. Feder Cooper, Christopher A. Choquette-Choo, Miranda Bogen +34
"Machine unlearning" is a popular proposed solution for mitigating the existence of content in an AI model that is problematic for legal or moral reasons, including privacy, copyri…
Measuring memorization in language models via probabilistic extraction
Jamie Hayes, Marika Swanberg, Harsh Chaudhari +6
Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable…
UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI
Ilia Shumailov, Jamie Hayes, Eleni Triantafillou +6
Exact unlearning was first introduced as a privacy mechanism that allowed a user to retract their data from machine learning models on request. Shortly after, inexact schemes were…
Beyond Slow Signs in High-fidelity Model Extraction
Hanna Foerster, Robert Mullins, Ilia Shumailov +1
Deep neural networks, costly to train and rich in intellectual property value, are increasingly threatened by model extraction attacks that compromise their confidentiality. Previo…
Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition
Eleni Triantafillou, Peter Kairouz, Fabian Pedregosa +12
We present the findings of the first NeurIPS competition on unlearning, which sought to stimulate the development of novel algorithms and initiate discussions on formal and robust…
Measuring memorization in RLHF for code completion
Aneesh Pappu, Billy Porter, Ilia Shumailov +1
Reinforcement learning with human feedback (RLHF) has become the dominant method to align large models to user preferences. Unlike fine-tuning, for which there are many studies reg…
Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
Jamie Hayes, Ilia Shumailov, Eleni Triantafillou +2
The high cost of model training makes it increasingly desirable to develop techniques for unlearning. These techniques seek to remove the influence of a training example without ha…