25 papers
SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring
Yuling Shi, Jinghan Xu, Kelin Fu +12
As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under se…
Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reports
Jingyi Chen, Songqiang Chen, Hengcheng Zhu +3
LLM agents span command-line interfaces (e.g., Codex) and agent frameworks (e.g., LangChain), integrating backend LLMs with harness code that parses model outputs, controls agent l…
Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills
Jialun Cao, Xinru Yan, Songqiang Chen +3
Software engineering (abbrev. SE) has continuously evolved through increasingly powerful forms of reuse, from source code and libraries to components and services. Recent advances…
Programming-by-Example for Batch-Editing Collision Meshes in 3D Software
Gengyang Xu, Dongwei Xiao, Hengcheng Zhu +4
As 3D software proliferates, software artifacts now extend beyond code and 2D user interfaces to include 3D assets. Among these assets, collision meshes are critical as they define…
Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware
Zimo Ji, Congying Xu, Zongjie Li +4
LLM coding agents increasingly rely on third-party agent skills from public marketplaces, which execute with the agent's privileges and create a software supply-chain attack surfac…
Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions
Zimo Ji, Zekai Zhang, Congying Xu +4
LLM coding agents are increasingly deployed to act autonomously on real production infrastructure. They execute shell commands, modify repositories, and call operational APIs. Howe…