8 papers · 1 filter
Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents
Jingjie Ning, Shanshan Zhong, Xiaochuan Li +1
AI research agents combine public information and experimental feedback to produce measurable results. The Discovery Certification Protocol (DCP) turns an outcome claim into an exe…
The Implementation Lottery: Auditing Idea Reliability in Automated Research
Jingjie Ning, Shanshan Zhong, Xiaochuan Li +2
Automated research agents use program scores to judge ideas. We call variation in this evidence across implementations the implementation lottery. We introduce an Idea Reliability…
Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer
Jingjie Ning, Xiaochuan Li, Shanshan Zhong +2
Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged by its terminal pipeline. A terminal s…
Auto Research with Specialist Agents Develops Effective and Non-Trivial Training Recipes
Jingjie Ning, Xiaochuan Li, Ji Zeng +2
We study auto research as a closed empirical loop driven by external measurement. Each submitted trial carries a hypothesis, an executable code edit, an evaluator-owned outcome, an…
daVinci-Env: Open SWE Environment Synthesis at Scale
Dayuan Fu, Shenyu Wu, Yunze Wu +11
Training capable software engineering (SWE) agents demands large-scale, executable, and verifiable environments that provide dynamic feedback loops for iterative code editing, test…
ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development
Pengrui Lu, Shiqi Zhang, Yunzhong Hou +8
Recent coding agents can generate complete codebases from simple prompts, yet existing evaluations focus on issue-level bug fixing and lag behind end-to-end development. We introdu…
daVinci-Agency: Unlocking Long-Horizon Agency Data-Efficiently
Mohan Jiang, Dayuan Fu, Junhao Shi +8
While Large Language Models (LLMs) excel at short-term tasks, scaling them to long-horizon agentic workflows remains challenging. The core bottleneck lies in the scarcity of traini…
daVinci-Dev: Agent-native Mid-training for Software Engineering
Ji Zeng, Dayuan Fu, Tiantian Mi +14
Recently, the frontier of Large Language Model (LLM) capabilities has shifted from single-turn code generation to agentic software engineering-a paradigm where models autonomously…