10 papers
One Run Is Not an Idea: The Implementation Lottery in Automated Research
Jingjie Ning, Shanshan Zhong, Xiaochuan Li +2
The paper studies how automated research systems can draw misleading conclusions when they rely on a single implementation of an idea, introducing the concept of an "implementation…
Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer
Jingjie Ning, Xiaochuan Li, Shanshan Zhong +2
Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged by its terminal pipeline. A terminal s…
Auto Research with Specialist Agents Develops Effective and Non-Trivial Training Recipes
Jingjie Ning, Xiaochuan Li, Ji Zeng +2
We study auto research as a closed empirical loop driven by external measurement. Each submitted trial carries a hypothesis, an executable code edit, an evaluator-owned outcome, an…
Pharos-ESG: A Framework for Multimodal Parsing, Contextual Narration, and Hierarchical Labeling of ESG Report
Yan Chen, Yu Zou, Jialei Zeng +3
Environmental, Social, and Governance (ESG) principles are reshaping the foundations of global financial governance, transforming capital allocation architectures, regulatory frame…
daVinci-Env: Open SWE Environment Synthesis at Scale
Dayuan Fu, Shenyu Wu, Yunze Wu +11
Training capable software engineering (SWE) agents demands large-scale, executable, and verifiable environments that provide dynamic feedback loops for iterative code editing, test…
ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
Pengrui Lu, Shiqi Zhang, Yunzhong Hou +8
Recent coding agents can generate complete codebases from simple prompts, yet existing evaluations focus on issue-level bug fixing and lag behind end-to-end development. We introdu…