7 papers
Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation
Michele Tufano, James McClure, José Cambronero +7
LLM-based agents are increasingly used for coding tasks, where they have outperformed many classical approaches and scaled to repository-level tasks, such as test generation. Howev…
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
Runxiang Cheng, Michele Tufano, José Cambronero +5
Bug Reproduction Tests (BRTs) have been used in many Automated Program Repair (APR) systems, primarily for validating promising fixes and aiding fix generation. In practice, when d…
Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair
José Cambronero, Michele Tufano, Sherry Shi +7
Agentic Automated Program Repair (APR) is increasingly tackling complex, repository-level bugs in industry, but ultimately these patches still need to be reviewed by a human before…
Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge
Sherry Shi, Renyao Wei, Michele Tufano +4
Reliable evaluation is crucial for advancing Automated Program Repair (APR), but prevailing benchmarks rely on execution-based evaluation methods (unit test pass@k), which fail to…
Towards Verified Code Reasoning by LLMs
Meghana Sistla, Gogul Balakrishnan, Pat Rondon +3
While LLM-based agents are able to tackle a wide variety of code reasoning questions, the answers are not always correct. This prevents the agent from being useful in situations wh…
Agentic Bug Reproduction for Effective Automated Program Repair at Google
Runxiang Cheng, Michele Tufano, Jürgen Cito +5
Bug reports often lack sufficient detail for developers to reproduce and fix the underlying defects. Bug Reproduction Tests (BRTs), tests that fail when the bug is present and pass…