7 papers
DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
Alexander D. Goldie, Zilin Wang, Adrian Hayler +17
Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems ha…
Investigating Memory in Model-Free RL with POPGym Arcade
Zekang Wang, Zhe He, Borong Zhang +2
How should we analyze memory in deep RL? We introduce tools for analyzing policies under partial observability and revealing how agents use memory to make decisions. To utilize the…
AIRA_2: Overcoming Bottlenecks in AI Research Agents
Karen Hambardzumyan, Nicolas Baldwin, Edan Toledo +22
Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the ben…
AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
Alisia Lupidi, Bhavul Gauri, Thomas Simon Foster +34
LLM agents hold significant promise for advancing scientific research. To accelerate this progress, we introduce AIRS-Bench (the AI Research Science Benchmark), a suite of 20 tasks…
What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
Alexis Audran-Reiss, Jordi Armengol-Estapé, Karen Hambardzumyan +17
AI research agents offer the promise to accelerate scientific progress by automating the design, implementation, and training of machine learning models. However, the field is stil…
AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench
Edan Toledo, Karen Hambardzumyan, Martin Josifoski +22
AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus o…