6 papers
Efficient SWE Agent Benchmarking via Trajectory-Aware Evaluation
Kefeng Duan, Dewu Zheng, Yanlin Wang +7
Evaluating software engineering agents on realistic benchmarks is costly, since each task may require multi-step code exploration, modification, and test execution. Existing effici…
Adaptive Critical Token-Aware Retrieval for Repository-Level Code Generation
Kefeng Duan, Dewu Zheng, Yanlin Wang +8
The repository-level code generation task requires synthesizing code that satisfies task requirements while remaining consistent with the target repository context. Since real-worl…
SWE-Prime: Fewer Trajectories, Better Performance
Dewu Zheng, Ruizhe Ye, Yanlin Wang +7
To improve large language models' ability to resolve real-world software issues, prior work has focused on constructing large-scale agent trajectory datasets and performing supervi…
From Static to Dynamic: Benchmarking Real-World Code Review with MCR-Bench
Dewu Zheng, Yanlin Wang, Xiwen Wang +5
In real-world software development, code review typically involves iterative interactions between developers and reviewers to improve software quality, making the process costly an…
Towards an Understanding of Context Utilization in Code Intelligence
Yanlin Wang, Kefeng Duan, Dewu Zheng +9
Code intelligence is an emerging domain in software engineering, aiming to improve the effectiveness and efficiency of various code-related tasks. Recent research suggests that inc…
Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
Dewu Zheng, Yanlin Wang, Ensheng Shi +4
With the rapid advancement of large language models (LLMs), extensive research has been conducted to investigate the code generation capabilities of LLMs. However, existing efforts…