3 citations · 7 across the 22 of their papers we have counts for
22 papers · 1 filter
OS-Bench: Evaluating SWE Agents on Cross-OS Code Migration
Qirui Jin, Kenan Li, Liao Zhu +8
Language-model (LM) agents are increasingly evaluated on repository-level software engineering (SWE) benchmarks derived from GitHub issues and pull requests. Although these benchma…
Update from Hell: Can Coding Agents Survive Hidden Breakage in Dependency Upgrades?
Zijian Luo, Runzhi He, Pengfei Gao +6
Modern software systems rely heavily on third-party dependencies, but upgrading those dependencies remains a costly maintenance activity. Dependency upgrades do not always preserve…
DPIAgent: Divide, Protocol, Isolate for Agentic Reproduction Test Generation
Hao Liu, Steven Liu, Xin Zhang +8
Reproduction test generation, producing a failing-then-passing test that captures a reported bug, is a critical step in automated software engineering. Existing agentic methods tre…
LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation
Han Li, Zhemin Fang, Rili Feng +8
Coding agent infrastructure is shifting from harness engineering toward loop engineering as coding agents are deployed for sustained long-horizon software development. Existing ben…
Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments
Haomin Qi, Xingliang Wang, Xuanqi Gao +9
Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a…
Can Large Language Models Generate Observability-Aware Code?
Yongliang Tao, Hongyu Zhang, Pengfei Gao +8
Recent advances in coding agents have enabled the generation of increasingly complex software systems. While existing evaluations primarily focus on functional correctness, product…
Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
Xinchen Wang, Ruida Hu, Cuiyun Gao +2
Software documentation is crucial for repository comprehension. While Large Language Models (LLMs) advance documentation generation from code snippets to entire repositories, exist…
Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice
Ruida Hu, Xinchen Wang, Xin-Cheng Wen +5
Code review is a cornerstone of software quality assurance, and recent advances in Large Language Models (LLMs) have shown promise in its automation. However, existing benchmarks f…
More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents
Pengfei Gao, Chao Peng
LLM-powered coding agents, which operate in iterative loops (turns) to solve software engineering tasks, are becoming increasingly powerful. However, their practical deployment is…
Extending Fill-In-the-Middle with Instructions for Steerable Code Completion
Zhensu Sun, Chengran Yang, Chao Peng +4
Code completion models often fail when the developer's intent is under-specified in the code context. To mitigate this, developers frequently use natural language comments to clari…
Reducing Cost of LLM Agents with Trajectory Reduction
Yuan-An Xiao, Pengfei Gao, Chao Peng +1
Multi-turn agent systems based on Large Language Models (LLMs) have become increasingly popular for software engineering tasks. While LLM agents demonstrate promising effectiveness…
Tool-integrated Reinforcement Learning for Repo Deep Search
Zexiong Ma, Chao Peng, Qunhong Zeng +3
Issue localization, the process of identifying code locations that need modification to resolve software issues, is a critical yet challenging task in software development. The sem…
Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
Trae Research Team, Pengfei Gao, Zhao Tian +12
Software issue resolution is a critical challenge in software engineering and has garnered increasing attention in recent years. With the rapid advancement of large language models…
CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
Xinchen Wang, Pengfei Gao, Chao Peng +2
Large language models (LLMs) have demonstrated strong capabilities in code generation, underscoring the critical need for rigorous and comprehensive evaluation. Existing evaluation…
SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
Zexiong Ma, Chao Peng, Pengfei Gao +3
Mainstream issue-resolving frameworks predominantly rely on commercial models, leading to high costs and privacy concerns. Existing training approaches for issue resolving struggle…
CoReQA: Uncovering Potentials of Language Models in Code Repository Question Answering
Jialiang Chen, Kaifa Zhao, Jie Liu +6
Large language models that enhance software development tasks, such as code generation, code completion, and code question answering (QA), have been extensively studied in both aca…
CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
Ruida Hu, Chao Peng, Jingyi Ren +6
In this work, we introduce CodeRepoQA, a large-scale benchmark specifically designed for evaluating repository-level question-answering capabilities in the field of software engine…
A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
Ruida Hu, Chao Peng, Jingyi Ren +6
Automatically resolving software issues is crucial for software development in practice, impacting the software quality and user experience. The process of resolving real-world iss…
AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
Xinchen Wang, Pengfei Gao, Xiangxin Meng +4
In software maintenance, bug reproduction is essential for effective fault localization and repair. Manually writing reproduction scripts is a time-consuming task with high require…
An Empirical Study on LLM-based Agents for Automated Bug Fixing
Xiangxin Meng, Zexiong Ma, Pengfei Gao +1
Large language models (LLMs) and LLM-based Agents have been applied to fix bugs automatically, demonstrating the capability in addressing software defects by engaging in developmen…
MarsCode Agent: AI-native Automated Bug Fixing
Yizhou Liu, Pengfei Gao, Xinchen Wang +4
Recent advances in large language models (LLMs) have shown significant potential to automate various software development tasks, including code completion, test generation, and bug…
RepoMasterEval: Evaluating Code Completion via Real-World Repositories
Qinyun Wu, Chao Peng, Pengfei Gao +9
With the growing reliance on automated code completion tools in software development, the need for comprehensive evaluation benchmarks has become critical. Existing benchmarks focu…