21 citations · 39 across the 8 of their papers we have counts for
9 papers
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Davide Paglieri, Bartłomiej Cupiał, Samuel Coward +10
Large Language Models (LLMs) and Vision Language Models (VLMs) possess extensive knowledge and exhibit promising reasoning abilities, however, they still struggle to perform well i…
Debating with More Persuasive LLMs Leads to More Truthful Answers
Akbir Khan, John Hughes, Dan Valentine +7
Common methods for aligning large language models (LLMs) with desired behaviour heavily rely on human-labelled data. However, as models grow increasingly sophisticated, they will s…
Leading the Pack: N-player Opponent Shaping
Alexandra Souly, Timon Willi, Akbir Khan +4
Reinforcement learning solutions have great success in the 2-player general sum setting. In this setting, the paradigm of Opponent Shaping (OS), in which agents account for the lea…
Scaling Opponent Shaping to High Dimensional Games
Akbir Khan, Timon Willi, Newton Kwan +5
In multi-agent settings with mixed incentives, methods developed for zero-sum games have been shown to lead to detrimental outcomes. To address this issue, opponent shaping (OS) me…
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
Alexander Rutherford, Benjamin Ellis, Matteo Gallici +18
Benchmarks are crucial in the development of machine learning algorithms, with available environments significantly influencing reinforcement learning (RL) research. Traditionally,…
MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
Mikayel Samvelyan, Akbir Khan, Michael Dennis +5
Open-ended learning methods that automatically generate a curriculum of increasingly challenging tasks serve as a promising avenue toward generally capable reinforcement learning a…
The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs
Laura Ruis, Akbir Khan, Stella Biderman +3
Despite widespread use of LLMs as conversational agents, evaluations of performance fail to capture a crucial aspect of communication: interpreting language in context -- incorpora…
Latent Racial Bias -- Evaluating Racism in Police Stop-and-Searches
Akbir Khan
In this paper, we introduce the latent racial bias, a metric and method to evaluate the racial bias within specific events. For the purpose of this paper we explore the British Hom…
Considering Race a Problem of Transfer Learning
Akbir Khan, Marwa Mahmoud
As biometric applications are fielded to serve large population groups, issues of performance differences between individual sub-groups are becoming increasingly important. In this…