1 citations · 1 across the 7 of their papers we have counts for
9 papers
The belief-action gap in language model agents
Ayse Sila Okcu, Weisheng Wang, Tongmeng Xie +4
Language model (LM) agents can fail for different reasons, requiring different explanations and interventions. We study how state beliefs, action values, and action preferences int…
Surprisal Minimisation over Goal-directed Alternatives Predicts Production Choice in Dialogue
Tom Utting, Mario Giulianelli, Arabella Sinclair
We model utterance production as probabilistic cost-sensitive choice over contextual alternatives, using information-theoretic notions of cost. We distinguish between goal-directed…
Probing for Reading Times
Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re +4
Probing has shown that language model representations encode rich linguistic information, but it remains unclear whether they also capture cognitive signals about human processing.…
Reasoning aligns language models to human cognition
Gonçalo Guiomar, Elia Torre, Pehuen Moure +4
Do language models make decisions under uncertainty like humans do, and what role does chain-of-thought (CoT) reasoning play in the underlying decision process? We introduce an act…
A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
Raghu Arghal, Fade Chen, Niall Dalton +6
Understanding an agent's goals helps explain and predict its behaviour, yet there is no established methodology for reliably attributing goals to agentic systems. We propose a fram…
Structure-Conditional Minimum Bayes Risk Decoding
Bryan Eikema, Anna Rutkiewicz, Mario Giulianelli
Minimum Bayes Risk (MBR) decoding has seen renewed interest as an alternative to traditional generation strategies. While MBR has proven effective in machine translation, where the…
Establishing Best Practices for Building Rigorous Agentic Benchmarks
Yuxuan Zhu, Tengjun Jin, Yada Pruksachatkun +22
Benchmarks are essential for quantitatively tracking progress in AI. As AI agents become increasingly capable, researchers and practitioners have introduced agentic benchmarks to e…
Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language
Christopher Summerfield, Lennart Luettgau, Magda Dubois +9
We examine recent research that asks whether current AI systems may be developing a capacity for "scheming" (covertly and strategically pursuing misaligned goals). We compare curre…
Skewed Score: A statistical framework to assess autograders
Magda Dubois, Harry Coppock, Mario Giulianelli +3
The evaluation of large language model (LLM) outputs is increasingly performed by other LLMs, a setup commonly known as "LLM-as-a-judge", or autograders. While autograders offer a…