7 papers
Towards Automatically Pruning Logging Code with Coding Agents: How Far Are We?
He Yang Yuan, Haonan Zhang, Xin Wang +4
Logging code supports debugging, monitoring, and software maintenance, but excessive logging can add noise, impose runtime overhead, and obscure diagnostic information. While prior…
SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements
Pengyu Xue, He Yang Yuan, Xin Wang +6
Although coding agents have achieved impressive performance on correctness-oriented benchmarks, their ability to make behavior-preserving non-functional improvements (NFIs) remains…
To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
Zhihao Lin, Junhua Zhu, Mingyi Zhou +5
LLM-based agents for program repair are increasingly built on a "generate-run-revise" paradigm, iteratively executing tests to evaluate and refine patches. This execution-based app…
Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement
Xin Wang, Liangtai Sun, Yaoming Zhu +8
Existing code-generation benchmarks score a single mapping from a complete prompt to a one-shot output. However, real web development is different. Users seldom write a full spec a…
Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs
He Yang Yuan, Xin Wang, Kundi Yao +3
Logging code plays an important role in software systems by recording key events and behaviors, which are essential for debugging and monitoring. However, insecure logging practice…
LLM4Perf: Large Language Models Are Effective Samplers for Multi-Objective Performance Modeling
Xin Wang, Zhenhao Li, Zishuo Ding
The performance of modern software systems is critically dependent on their complex configuration options. Building accurate performance models to navigate this vast space requires…
Defects4Log: Benchmarking LLMs for Logging Code Defect Detection and Reasoning
Xin Wang, Zhenhao Li, Zishuo Ding
Logging code is written by developers to capture system runtime behavior and plays a vital role in debugging, performance analysis, and system monitoring. However, defects in loggi…