4 papers
LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems
Yun Peng, Zihan Wu, Zeyang Zhuang +6
Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks with large reference implementati…
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
Xin Zhou, Chun Yong Chong, Kisub Kim +11
Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs. Yet existing repository-level…
What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agents
Rui Shu, Chun Yong Chong, Xin Zhou +6
Coding agents are ranked almost entirely by resolve rate: whether their final patch passes the target tests. Yet two agents can reach the same outcome through very different proces…
ComBench: A Repo-level Real-world Benchmark for Compilation Error Repair
Jia Li, Zeyang Zhuang, Zhuangbin Chen +3
Compilation errors pose pervasive and critical challenges in software development, significantly hindering productivity. Therefore, Automated Compilation Error Repair (ACER) techni…