95 citations · 102 across the 6 of their papers we have counts for
7 papers
The AI Risk Spectrum: From Dangerous Capabilities to Existential Threats
Markov Grey, Charbel-Raphaël Segerie
As AI systems become more capable, integrated, and widespread, understanding the associated risks becomes increasingly important. This paper maps the full spectrum of AI risks, fro…
The bitter lesson of misuse detection
Hadrien Mariaccia, Charbel-Raphaël Segerie, Diego Dorn
Prior work on jailbreak detection has established the importance of adversarial robustness for LLMs but has largely focused on the model ability to resist adversarial inputs and to…
Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods
Markov Grey, Charbel-Raphaël Segerie
As frontier AI systems advance toward transformative capabilities, we need a parallel transformation in how we measure and evaluate these systems to ensure safety and inform govern…
In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
Ben Bucknall, Saad Siddiqui, Lara Thurnherr +19
International cooperation is common in AI research, including between geopolitical rivals. While many experts advocate for greater international cooperation on AI safety to address…
BELLS: A Framework Towards Future Proof Benchmarks for the Evaluation of LLM Safeguards
Diego Dorn, Alexandre Variengien, Charbel-Raphaël Segerie +1
Input-output safeguards are used to detect anomalies in the traces produced by Large Language Models (LLMs) systems. These detectors are at the core of diverse safety-critical appl…
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Stephen Casper, Xander Davies, Claudia Shi +29
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals. RLHF has emerged as the central method used to finetune state-of…
Continuous Homeostatic Reinforcement Learning for Self-Regulated Autonomous Agents
Hugo Laurençon, Charbel-Raphaël Ségerie, Johann Lussange +1
Homeostasis is a prevalent process by which living beings maintain their internal milieu around optimal levels. Multiple lines of evidence suggest that living beings learn to act t…