1 citations · 1 across the 7 of their papers we have counts for
7 papers
Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results
Jan Batzner, Sree Harsha Nelaturu, Damian Stachura +45
AI evaluations are widely used for testing and understanding progress. However, the diverse evaluators bring with them inconsistencies that challenge analysis and comparison. First…
Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
Srujan P Mule, Aniketh Garikaparthi, Manasi Patwardhan
As language models accelerate scientific research by automating hypothesis generation and implementation, a new bottleneck emerges: evaluating and filtering hundreds of AI-generate…
ResearchGym: Evaluating Language Model Agents on Real-World AI Research
Aniketh Garikaparthi, Manasi Patwardhan, Arman Cohan
We introduce ResearchGym, a benchmark and execution environment for evaluating AI agents on end-to-end research. To instantiate this, we repurpose five oral and spotlight papers fr…
Can LLMs Perceive Time? An Empirical Investigation
Aniketh Garikaparthi
Large language models cannot estimate how long their own tasks take. We investigate this limitation through four experiments across 68 tasks and four model families. Pre-task estim…
REVERE: Reflective Evolving Research Engineer
Balaji Dinesh Gangireddi, Aniketh Garikaparthi, Manasi Patwardhan +1
Existing prompt-optimization techniques rely on local signals, causing poor generalization across tasks. In addition, they also rely on weak update mechanisms, such as full-prompt…
MIR: Methodology Inspiration Retrieval for Scientific Research Problems
Aniketh Garikaparthi, Manasi Patwardhan, Aditya Sanjiv Kanade +3
There has been a surge of interest in harnessing the reasoning capabilities of Large Language Models (LLMs) to accelerate scientific discovery. While existing approaches rely on gr…
IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery
Aniketh Garikaparthi, Manasi Patwardhan, Lovekesh Vig +1
The rapid advancement in capabilities of large language models (LLMs) raises a pivotal question: How can LLMs accelerate scientific discovery? This work tackles the crucial first s…