6 papers
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Michele Tufano, James McClure, José Cambronero +7
LLM-based agents are increasingly used for coding tasks, where they have outperformed many classical approaches and scaled to repository-level tasks, such as test generation. Howev…
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Runxiang Cheng, Michele Tufano, José Cambronero +7
Bug Reproduction Tests (BRTs) have been used in many Automated Program Repair (APR) systems, primarily for validating fixes and aiding fix generation. In practice, when developers…
Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair
José Cambronero, Michele Tufano, Sherry Shi +7
Agentic Automated Program Repair (APR) is increasingly tackling complex, repository-level bugs in industry, but ultimately these patches still need to be reviewed by a human before…
Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge
Sherry Shi, Renyao Wei, Michele Tufano +4
Reliable evaluation is crucial for advancing Automated Program Repair (APR), but prevailing benchmarks rely on execution-based evaluation methods (unit test pass@k), which fail to…
Agentic Bug Reproduction for Effective Automated Program Repair at Google
Runxiang Cheng, Michele Tufano, Jürgen Cito +5
Bug reports often lack sufficient detail for developers to reproduce and fix the underlying defects. Bug Reproduction Tests (BRTs), tests that fail when the bug is present and pass…
Evaluating Agent-based Program Repair at Google
Pat Rondon, Renyao Wei, José Cambronero +5
Agent-based program repair offers to automatically resolve complex bugs end-to-end by combining the planning, tool use, and code generation abilities of modern LLMs. Recent work ha…