27 citations · 110 across the 41 of their papers we have counts for
4 papers · 1 filter
A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness
Leo Schwinn, Moritz Ladenburger, Tim Beyer +3
Automated \enquote{LLM-as-a-Judge} frameworks have become the de facto standard for scalable evaluation across natural language processing. For instance, in safety evaluation, thes…
Jailbreak Distillation: Renewable Safety Benchmarking
Jingyu Zhang, Ahmed Elgohary, Xiawei Wang +5
Large language models (LLMs) are rapidly deployed in critical applications, raising urgent needs for robust safety benchmarking. We propose Jailbreak Distillation (JBDistill), a no…
A Generative Approach to LLM Harmfulness Mitigation with Red Flag Tokens
David Dobre, Mehrnaz Mofakhami, Sophie Xhonneux +2
Many safety post-training methods for large language models (LLMs) are designed to modify the model's behaviour from producing unsafe answers to issuing refusals. However, such dis…
Extracting Unlearned Information from LLMs with Activation Steering
Atakan Seyitoğlu, Aleksei Kuvshinov, Leo Schwinn +1
An unintended consequence of the vast pretraining of Large Language Models (LLMs) is the verbatim memorization of fragments of their training data, which may contain sensitive or c…