5 papers
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Michele Tufano, James McClure, José Cambronero +7
LLM-based agents are increasingly used for coding tasks, where they have outperformed many classical approaches and scaled to repository-level tasks, such as test generation. Howev…
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Runxiang Cheng, Michele Tufano, José Cambronero +7
Bug Reproduction Tests (BRTs) have been used in many Automated Program Repair (APR) systems, primarily for validating fixes and aiding fix generation. In practice, when developers…
Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair
José Cambronero, Michele Tufano, Sherry Shi +7
Agentic Automated Program Repair (APR) is increasingly tackling complex, repository-level bugs in industry, but ultimately these patches still need to be reviewed by a human before…
Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge
Sherry Shi, Renyao Wei, Michele Tufano +4
Reliable evaluation is crucial for advancing Automated Program Repair (APR), but prevailing benchmarks rely on execution-based evaluation methods (unit test pass@k), which fail to…
Agentic Bug Reproduction for Effective Automated Program Repair at Google
Runxiang Cheng, Michele Tufano, Jürgen Cito +5
Bug reports often lack sufficient detail for developers to reproduce and fix the underlying defects. Bug Reproduction Tests (BRTs), tests that fail when the bug is present and pass…