8 papers
REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
Smriti Jha, Matteo Paltenghi, Chandra Maddila +3
Production deployment of AI coding agents requires fast, reproducible evaluation signals. Existing industrial practices trade off speed and fidelity: online A/B testing takes weeks…
Agentic Code Reasoning
Shubham Ugare, Satish Chandra
Can LLM agents explore codebases and reason about code semantics without executing the code? We study this capability, which we call agentic code reasoning, and introduce semi-form…
Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair
José Cambronero, Michele Tufano, Sherry Shi +7
Agentic Automated Program Repair (APR) is increasingly tackling complex, repository-level bugs in industry, but ultimately these patches still need to be reviewed by a human before…
Towards Verified Code Reasoning by LLMs
Meghana Sistla, Gogul Balakrishnan, Pat Rondon +3
While LLM-based agents are able to tackle a wide variety of code reasoning questions, the answers are not always correct. This prevents the agent from being useful in situations wh…
Natural Language Outlines for Code: Literate Programming in the LLM Era
Kensen Shi, Deniz Altınbüken, Saswat Anand +17
We propose using natural language outlines as a novel modality and interaction surface for providing AI assistance to developers throughout the software development process. An NL…
Agentic Bug Reproduction for Effective Automated Program Repair at Google
Runxiang Cheng, Michele Tufano, Jürgen Cito +5
Bug reports often lack sufficient detail for developers to reproduce and fix the underlying defects. Bug Reproduction Tests (BRTs), tests that fail when the bug is present and pass…