8 papers
InnovatorBench: Evaluating Agents' Ability to Conduct Innovative LLM Research
Yunze Wu, Dayuan Fu, Weiye Si +13
AI agents could accelerate scientific discovery by automating hypothesis formation, experiment design, coding, execution, and analysis, yet existing benchmarks probe narrow skills…
DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery
Keyu Li, Mohan Jiang, Dayuan Fu +4
The rapid advancement of large language models has fundamentally shifted the bottleneck in AI development from computational power to data availability-with countless valuable data…
AlphaGo Moment for Model Architecture Discovery
Yixiu Liu, Yang Nan, Weixian Xu +4
While AI systems demonstrate exponentially improving capabilities, the pace of AI research itself remains linearly bounded by human cognitive capacity, creating an increasingly sev…
ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquiry
Tianze Xu, Pengrui Lu, Lyumanshan Ye +2
The emergence of deep research systems presents significant capabilities in problem-solving, extending from basic queries to sophisticated research tasks. However, existing benchma…
Interaction as Intelligence: Deep Research With Human-AI Partnership
Lyumanshan Ye, Xiaojie Cai, Xinkai Wang +23
This paper introduces "Interaction as Intelligence" research series, presenting a reconceptualization of human-AI relationships in deep research tasks. Traditional approaches treat…
Generative AI Act II: Test Time Scaling Drives Cognition Engineering
Shijie Xia, Yiwei Qin, Xuefeng Li +11
The first generation of Large Language Models - what might be called "Act I" of generative AI (2020-2023) - achieved remarkable success through massive parameter and data scaling,…