5 papers
Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
Davide Paglieri, BartÅomiej CupiaÅ, Jonathan Cook +6
Training large language models (LLMs) to reason via reinforcement learning (RL) significantly improves their problem-solving capabilities. In agentic settings, existing methods lik…
Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments
Romain Froger, Pierre Andrews, Matteo Bettini +21
We introduce Gaia2, a benchmark for evaluating large language model agents in realistic, asynchronous environments. Unlike prior static or synchronous evaluations, Gaia2 introduces…
ARE: Scaling Up Agent Environments and Evaluations
Romain Froger, Pierre Andrews, Matteo Bettini +21
We introduce Meta Agents Research Environments (ARE), a research platform for scalable creation of environments, integration of synthetic or real applications, and execution of age…
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Davide Paglieri, BartÅomiej CupiaÅ, Samuel Coward +10
Large Language Models (LLMs) and Vision Language Models (VLMs) possess extensive knowledge and exhibit promising reasoning abilities, however, they still struggle to perform well i…
Training Language Models on Synthetic Edit Sequences Improves Code Synthesis
Ulyana Piterbarg, Lerrel Pinto, Rob Fergus
Software engineers mainly write code by editing existing programs. In contrast, language models (LMs) autoregressively synthesize programs in a single pass. One explanation for thi…