4 papers
Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions
Zimo Ji, Zekai Zhang, Congying Xu +4
LLM coding agents are increasingly deployed to act autonomously on real production infrastructure. They execute shell commands, modify repositories, and call operational APIs. Howe…
Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation
Zongyi Lyu, Songqiang Chen, Zhenlan Ji +5
Multi-agent systems (MASs) have emerged as a promising paradigm for automated code generation, demonstrating impressive performance on established benchmarks. Despite their prosper…
Metamorphic Testing for Audio Content Moderation Software
Wenxuan Wang, Yongjiang Wu, Junyuan Zhang +5
The rapid growth of audio-centric platforms and applications such as WhatsApp and Twitter has transformed the way people communicate and share audio content in modern society. Howe…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…