1 citations · 1 across the 7 of their papers we have counts for
4 papers · 1 filter
Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results
Jan Batzner, Sree Harsha Nelaturu, Damian Stachura +45
AI evaluations are widely used for testing and understanding progress. However, the diverse evaluators bring with them inconsistencies that challenge analysis and comparison. First…
ResearchGym: Evaluating Language Model Agents on Real-World AI Research
Aniketh Garikaparthi, Manasi Patwardhan, Arman Cohan
We introduce ResearchGym, a benchmark and execution environment for evaluating AI agents on end-to-end research. To instantiate this, we repurpose five oral and spotlight papers fr…
MIR: Methodology Inspiration Retrieval for Scientific Research Problems
Aniketh Garikaparthi, Manasi Patwardhan, Aditya Sanjiv Kanade +3
There has been a surge of interest in harnessing the reasoning capabilities of Large Language Models (LLMs) to accelerate scientific discovery. While existing approaches rely on gr…
IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery
Aniketh Garikaparthi, Manasi Patwardhan, Lovekesh Vig +1
The rapid advancement in capabilities of large language models (LLMs) raises a pivotal question: How can LLMs accelerate scientific discovery? This work tackles the crucial first s…