5 papers · 1 filter
The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections
Milad Nasr, Nicholas Carlini, Chawin Sitawarin +11
How should we evaluate the robustness of language model defenses? Current defenses against jailbreaks and prompt injections (which aim to prevent an attacker from eliciting harmful…
Text-to-Image Models Leave Identifiable Signatures: Implications for Leaderboard Security
Ali Naseh, Anshuman Suri, Yuefeng Peng +3
Generative AI leaderboards are central to evaluating model capabilities, but remain vulnerable to manipulation. Among key adversarial objectives is rank manipulation, where an atta…
Cascading Adversarial Bias from Injection to Distillation in Language Models
Harsh Chaudhari, Jamie Hayes, Matthew Jagielski +3
Model distillation has become essential for creating smaller, deployable language models that retain larger system capabilities. However, widespread deployment raises concerns abou…
Exploiting Leaderboards for Large-Scale Distribution of Malicious Models
Anshuman Suri, Harsh Chaudhari, Yuefeng Peng +3
While poisoning attacks on machine learning models have been extensively studied, the mechanisms by which adversaries can distribute poisoned models at scale remain largely unexplo…
Measuring memorization in language models via probabilistic extraction
Jamie Hayes, Marika Swanberg, Harsh Chaudhari +6
Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable…