4 papers · 1 filter
AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
Alisia Lupidi, Bhavul Gauri, Thomas Simon Foster +34
LLM agents hold significant promise for advancing scientific research. To accelerate this progress, we introduce AIRS-Bench (the AI Research Science Benchmark), a suite of 20 tasks…
Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments
Romain Froger, Pierre Andrews, Matteo Bettini +21
We introduce Gaia2, a benchmark for evaluating large language model agents in realistic, asynchronous environments. Unlike prior static or synchronous evaluations, Gaia2 introduces…
ARE: Scaling Up Agent Environments and Evaluations
Romain Froger, Pierre Andrews, Matteo Bettini +21
We introduce Meta Agents Research Environments (ARE), a research platform for scalable creation of environments, integration of synthetic or real applications, and execution of age…
What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
Alexis Audran-Reiss, Jordi Armengol-Estapé, Karen Hambardzumyan +17
AI research agents offer the promise to accelerate scientific progress by automating the design, implementation, and training of machine learning models. However, the field is stil…