23 citations · 35 across the 12 of their papers we have counts for
14 papers
Single-turn emergency psychiatric triage across 15 frontier AI chatbots
Veith Weilnhammer, Lennart Luettgau, Christopher Summerfield +5
People increasingly turn to general-purpose AI chatbots for advice about emotional and mental health problems, but the ability of these systems to recognize and appropriately triag…
Seven simple steps for log analysis in AI systems
Magda Dubois, Ekin Zorer, Maia Hamin +17
AI systems produce large volumes of logs as they interact with tools and users. Analysing these logs can help understand model capabilities, propensities, and behaviours, or assess…
When Do LLM Preferences Predict Downstream Behavior?
Katarina Slama, Alexandra Souly, Dishank Bansal +3
Preference-driven behavior in LLMs may be a necessary precondition for AI misalignment such as sandbagging: models cannot strategically pursue misaligned goals unless their behavio…
Ask don't tell: Reducing sycophancy in large language models
Magda Dubois, Cozmin Ududec, Christopher Summerfield +1
Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-sta…
A clinically validated framework for auditing AI chatbot behavior in mental health interactions
Veith Weilnhammer, Kevin YC Hou, Lennart Luettgau +3
Millions of users turn to consumer AI chatbots to discuss emotional, behavioral, and mental-health concerns, creating an urgent need for rigorous and scalable safety evaluations. H…
Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationships
Hannah Rose Kirk, Henry Davidson, Ed Saunders +4
Humans are increasingly forming parasocial relationships with AI systems, and modern AI shows an increasing tendency to display social and relationship-seeking behaviour. However,…
Measuring what Matters: Construct Validity in Large Language Model Benchmarks
Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou +39
Evaluating large language models (LLMs) is crucial for both assessing their capabilities and identifying safety or robustness issues prior to deployment. Reliably measuring abstrac…
People readily follow personal advice from AI but it does not improve their well-being
Lennart Luettgau, Vanessa Cheung, Magda Dubois +8
People increasingly seek personal advice from large language models (LLMs), yet whether humans follow their advice, and its consequences for their well-being, remains unknown. In a…
Conversational AI increases political knowledge as effectively as self-directed internet search
Lennart Luettgau, Hannah Rose Kirk, Kobi Hackenburg +6
Conversational AI systems are increasingly being used in place of traditional search engines to help users complete information-seeking tasks. This has raised concerns in the polit…
Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language
Christopher Summerfield, Lennart Luettgau, Magda Dubois +9
We examine recent research that asks whether current AI systems may be developing a capacity for "scheming" (covertly and strategically pursuing misaligned goals). We compare curre…
Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness
Sebastian Dohnány, Zeb Kurth-Nelson, Eleanor Spens +6
Artificial intelligence chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread socia…
Skewed Score: A statistical framework to assess autograders
Magda Dubois, Harry Coppock, Mario Giulianelli +3
The evaluation of large language model (LLM) outputs is increasingly performed by other LLMs, a setup commonly known as "LLM-as-a-judge", or autograders. While autograders offer a…
HiBayES: A Hierarchical Bayesian Modeling Framework for AI Evaluation Statistics
Lennart Luettgau, Harry Coppock, Magda Dubois +2
As Large Language Models (LLMs) and other AI systems evolve, robustly estimating their capabilities from inherently stochastic outputs while systematically quantifying uncertainty…
Replay and compositional computation
Zeb Kurth-Nelson, Timothy Behrens, Greg Wayne +5
Replay in the brain has been viewed as rehearsal, or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form…