7 citations · 12 across the 13 of their papers we have counts for
16 papers
SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning
Kaiwen Zhou, Ahmed Elgohary, A S M Iftekhar +1
The ability of LLM agents to plan and invoke tools exposes them to new safety risks, making a comprehensive red-teaming system crucial for discovering vulnerabilities and ensuring…
AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench
Edan Toledo, Karen Hambardzumyan, Martin Josifoski +22
AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus o…
The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements
Bingchen Zhao, Despoina Magka, Minqi Jiang +20
Rapid advancements in large language models (LLMs) have the potential to assist in scientific progress. A critical capability toward this endeavor is the ability to reproduce exist…
Ad-Hoc Human-AI Coordination Challenge
Tin Dizdarević, Ravi Hammond, Tobias Gessler +7
Achieving seamless coordination between AI agents and humans is crucial for real-world applications, yet it remains a significant open challenge. Hanabi is a cooperative card game…
The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind
Andrei Lupu, Timon Willi, Jakob Foerster
As Large Language Models (LLMs) gain agentic abilities, they will have to navigate complex multi-agent scenarios, interacting with human users and other agents in cooperative and c…
OvercookedV2: Rethinking Overcooked for Zero-Shot Coordination
Tobias Gessler, Tin Dizdarevic, Ani Calinescu +3
AI agents hold the potential to transform everyday life by helping humans achieve their goals. To do this successfully, agents need to be able to coordinate with novel partners wit…
Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps
Benjamin Ellis, Matthew T. Jackson, Andrei Lupu +4
In reinforcement learning (RL), it is common to apply techniques used broadly in machine learning such as neural network function approximators and momentum-based optimizers. Howev…
CURATe: Benchmarking Personalised Alignment of Conversational AI Assistants
Lize Alberts, Benjamin Ellis, Andrei Lupu +1
We introduce a multi-turn benchmark for evaluating personalised alignment in LLM-based AI assistants, focusing on their ability to handle user-provided safety-critical contexts. Ou…
The Llama 3 Herd of Models
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri +556
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models th…
Behaviour Distillation
Andrei Lupu, Chris Lu, Jarek Liesen +2
Dataset distillation aims to condense large datasets into a small number of synthetic examples that can be used as drop-in replacements when training new models. It has application…
Discovering Minimal Reinforcement Learning Environments
Jarek Liesen, Chris Lu, Andrei Lupu +3
Reinforcement learning (RL) agents are commonly trained and evaluated in the same environment. In contrast, humans often train in a specialized environment before being evaluated,…
Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts
Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu +9
As large language models (LLMs) become increasingly prevalent across many real-world applications, understanding and enhancing their robustness to adversarial attacks is of paramou…
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
Alexander Rutherford, Benjamin Ellis, Matteo Gallici +18
Benchmarks are crucial in the development of machine learning algorithms, with available environments significantly influencing reinforcement learning (RL) research. Traditionally,…
Self-Explaining Deviations for Coordination
Hengyuan Hu, Samuel Sokota, David Wu +4
Fully cooperative, partially observable multi-agent problems are ubiquitous in the real world. In this paper, we focus on a specific subclass of coordination problems in which huma…
Grounding Aleatoric Uncertainty for Unsupervised Environment Design
Minqi Jiang, Michael Dennis, Jack Parker-Holder +5
Adaptive curricula in reinforcement learning (RL) have proven effective for producing policies robust to discrepancies between the train and test environment. Recently, the Unsuper…
Option-Critic in Cooperative Multi-agent Systems
Jhelum Chakravorty, Nadeem Ward, Julien Roy +4
In this paper, we investigate learning temporal abstractions in cooperative multi-agent systems, using the options framework (Sutton et al, 1999). First, we address the planning pr…