18 papers
SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring
Yuling Shi, Jinghan Xu, Kelin Fu +12
As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under se…
Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills
Jialun Cao, Xinru Yan, Songqiang Chen +3
Software engineering (abbrev. SE) has continuously evolved through increasingly powerful forms of reuse, from source code and libraries to components and services. Recent advances…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…
iCoRe: An Iterative Correlation-Aware Retriever for Bug Reproduction Test Generation
Junyi Wang, Jialun Cao, Zhongxin Liu
Automatically generating bug reproduction tests (BRT) from issue descriptions is crucial for software maintenance. LLM-based approaches have shown great potential for this task. Th…
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
Yuanyi Wang, Yifan Yang, Su Lu +9
Continual post-training aims to extend large language models (LLMs) with new knowledge, skills, and behaviors, yet it remains unclear when sequential updates enable capability tran…
Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
Qiming Zhu, Jialun Cao, Xuanang Chen +6
Current research on large language models (LLMs) with retrieval-augmented code generation (RACG) has largely focused on single-language settings, leaving their cross-lingual effect…