14 papers
Chain-of-Thought Hijacking
Jianli Zhao, Tingchen Fu, Rylan Schaeffer +2
Large Reasoning Models (LRMs) improve task performance through extended inference-time reasoning. Although previous studies suggest that longer reasoning should lead to more robust…
Who's in Charge? Disempowerment Patterns in Real-World LLM Usage
Mrinank Sharma, Miles McCain, Raymond Douglas +1
Although AI assistants are now deeply embedded in society, there has been limited empirical study of how their usage affects human empowerment. We present the first large-scale emp…
Eliciting Harmful Capabilities by Fine-Tuning On Safeguarded Outputs
Jackson Kaunismaa, Avery Griffin, John Hughes +3
Model developers implement safeguards in frontier models to prevent misuse, for example, by employing classifiers to filter dangerous outputs. In this work, we demonstrate that eve…
Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks
Hoagy Cunningham, Jerry Wei, Zihan Wang +26
We introduce enhanced Constitutional Classifiers that deliver production-grade jailbreak robustness with dramatically reduced computational costs and refusal rates compared to prev…
Towards Safeguarding LLM Fine-tuning APIs against Cipher Attacks
Jack Youstra, Mohammed Mahfoud, Yang Yan +3
Large language model fine-tuning APIs enable widespread model customization, yet pose significant safety risks. Recent work shows that adversaries can exploit access to these APIs…
PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning
Tingchen Fu, Mrinank Sharma, Philip Torr +3
Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks. To address this concern, we introduce PoisonBenc…