7 papers · 1 filter
When Code Becomes an LLM Call: An Empirical Study of Call-Site Contracts in Agentic Systems
Shihao Weng, Longtian Wang, Yang Feng +6
Software quality has long rested on specifications that state what code must do, and on testing and verification that check that the code does it. Agentic systems change this found…
LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation
Han Li, Zhemin Fang, Rili Feng +8
Coding agent infrastructure is shifting from harness engineering toward loop engineering as coding agents are deployed for sustained long-horizon software development. Existing ben…
Failure as a Process: An Anatomy of CLI Coding Agent Trajectories
Xiangxin Zhao, Han Li, Shuaiting Li +4
Large language model (LLM) coding agents are increasingly deployed to autonomously perform software engineering tasks in terminal-based environments, making their reliability a gro…
An Iterative Test-and-Repair Framework for Competitive Code Generation
Lingxiao Tang, Muyang Ye, Zhaoyang Chu +4
Large language models (LLMs) have made remarkable progress in code generation, but competitive programming remains a challenge. Recent training-based methods have improved code gen…
HerAgent: Rethinking the Automated Environment Deployment via Hierarchical Test Pyramid
Xiang Li, Siyu Lu, Federica Sarro +2
Automated software environment setup is a prerequisite for testing, debugging, and reproducing failures, yet remains challenging in practice due to complex dependencies, heterogene…
Environment-in-the-Loop: Rethinking Code Migration with LLM-based Agents
Xiang Li, Zhiwei Fei, Ying Ma +3
Modern software systems continuously undergo code upgrades to enhance functionality, security, and performance, and Large Language Models (LLMs) have demonstrated remarkable capabi…
JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11)
Nishil Amin, Zhiwei Fei, Xiang Li +2
We build a benchmark to evaluate large language models (LLMs) for source code migration tasks, specifically upgrading functions from Java 8 to Java 11. We first collected a dataset…