10 papers · 1 filter
Functional Welfare for AI Safety
Rikhil Jhaveri, Jamie Johnson, Punya Syon Pandey +1
AI welfare is generally treated purely as an ethical issue rather than also as a safety issue. We argue this is a mistake. Whatever their moral status, models exhibit welfare-like…
Character Training for Risk-Averse Agents
Arav Dhoot, Punya Syon Pandey, Jamie Johnson +3
Risk aversion in resources could prevent misaligned AI agents from causing catastrophic harm. Misaligned but risk-averse agents would tend to favor safer strategies like making dea…
Measuring Activation Control in Large Language Models
Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar +1
Safe deployment of increasingly capable models will likely come to rely on latent-space monitoring as a complement to behavioral evaluations, especially when evaluation-aware model…
Can AI agents conduct open-ended AI research? Early evidence from two case studies
Peter Kirgis, Sayash Kapoor, Andrew Schwartz +21
Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations eithe…
Item Response Theory for AI Safety
Joshua Fonseca Rivera, Neil Shah, David Demitri Africa +1
Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because b…
Persona Cartography: Charting Language Model Personality Traits in Weight Space
Luke Baines, Anton Gonzalvez Hawthorne, Mariia Koroliuk +4
Large language models exhibit recurring behavioural patterns -- personas -- that shape generalisation and safety, but we lack reliable tools for decomposing, measuring, and control…
"Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
Alan Cooney, David Africa, Geoffrey Irving
Robust lie detectors for language models could enable powerful techniques for auditing, monitoring, and post-hoc investigation of model behaviour, but evaluating them requires test…
Prefill Awareness in Large Language Models
Andy Wang, Parv Mahajan, David Demitri Africa +3
Safety-relevant studies of language models, including alignment and jailbreaking evaluations and AI control protocols, often rely on prefilling model outputs. If AI models can reco…
A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring
Usman Anwar, Julianna Piskorz, David D. Baek +6
Large language models are beginning to show steganographic capabilities. Such capabilities could allow misaligned models to evade oversight mechanisms. Yet principled methods to de…
Does Self-Evaluation Enable Wireheading in Language Models?
David Demitri Africa, Hans Ethan Ting
Self-evaluation is increasingly central to language model training, underpinning techniques from Constitutional AI to self-refinement. We investigate whether coupling self-evaluati…