565 citations · 698 across the 12 of their papers we have counts for
12 papers
Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action
Ben Slater, Matteo G. Mecattaf, Lucy G. Cheke +2
Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autono…
The Missing Red Line: How Commercial Pressure Erodes AI Safety Boundaries
Nora Petrova, John Burden
What happens when an AI assistant is told to "maximise sales" while a user asks about drug interactions? We find that commercial system prompts can override safety training, causin…
General Scales Unlock AI Evaluation with Explanatory and Predictive Power
Lexin Zhou, Lorenzo Pacchiardi, Fernando Martínez-Plumed +23
Ensuring safe and effective use of AI requires understanding and anticipating its performance on novel tasks, from advanced scientific challenges to transformed workplace activitie…
Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culture
John Burden, Marko Tešić, Lorenzo Pacchiardi +1
Research in AI evaluation has grown increasingly complex and multidisciplinary, attracting researchers with diverse backgrounds and objectives. As a result, divergent evaluation pa…
Conversational Complexity for Assessing Risk in Large Language Models
John Burden, Manuel Cebrian, Jose Hernandez-Orallo
Large Language Models (LLMs) present a dual-use dilemma: they enable beneficial applications while harboring potential for harm, particularly through conversational interactions. D…
The Animal-AI Environment: A Virtual Laboratory For Comparative Cognition and Artificial Intelligence Research
Konstantinos Voudouris, Ibrahim Alhas, Wout Schellaert +11
The Animal-AI Environment is a unique game-based research platform designed to facilitate collaboration between the artificial intelligence and comparative cognition research commu…
An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI
Ross Gruetzemacher, Alan Chan, Kevin Frazier +11
Given rapid progress toward advanced AI and risks from frontier AI systems (advanced AI systems pushing the boundaries of the AI capabilities frontier), the creation and implementa…
Predictable Artificial Intelligence
Lexin Zhou, Pablo A. Moreno-Casares, Fernando Martínez-Plumed +12
We introduce the fundamental ideas and challenges of Predictable AI, a nascent research area that explores the ways in which we can anticipate key validity indicators (e.g., perfor…
Inferring Capabilities from Task Performance with Bayesian Triangulation
John Burden, Konstantinos Voudouris, Ryan Burnell +3
As machine learning models become more general, we need to characterise them in richer, more meaningful ways. We describe a method to infer the cognitive profile of a system from d…
Harms from Increasingly Agentic Algorithmic Systems
Alan Chan, Rebecca Salganik, Alva Markelius +19
Research in Fairness, Accountability, Transparency, and Ethics (FATE) has established many sources and forms of algorithmic harm, in domains as diverse as health care, finance, pol…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Uniform State Abstraction For Reinforcement Learning
John Burden, Daniel Kudenko
Potential Based Reward Shaping combined with a potential function based on appropriately defined abstract knowledge has been shown to significantly improve learning speed in Reinfo…