4 papers
A Model Organism for Studying Unverbalized Evaluation Awareness
Daniel Donnelly, Evan Chapple, Haritz Puerto +1
Large language models can recognize when they are being evaluated and change their behavior accordingly, undermining the reliability of safety benchmarks. Evaluation awareness (EA)…
SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents
Saswat Das, Parvati Viswanathan, Daniel Donnelly +3
Self-evolving LLM agents have gained prominence for their ability to improve after deployment by modifying their harness, including their controller instructions, memory management…
ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D
Lena Libon, Ben Rank, Jehyeok Yeon +5
As AI agents begin to automate AI R&D, we need ways to assess whether their outputs are safe to deploy, even when the agents themselves may be untrusted. AI control offers one such…
Expressive Reward Synthesis with the Runtime Monitoring Language
Daniel Donnelly, Angelo Ferrando, Francesco Belardinelli
A key challenge in reinforcement learning (RL) is reward (mis)specification, whereby imprecisely defined reward functions can result in unintended, possibly harmful, behaviours. In…