From the 1 of 6 linked papers with an AI index.
6 papers
SWE-NFI: Studying and Benchmarking Coding Agents for Non-Functional Improvements
Pengyu Xue, He Yang Yuan, Xin Wang +6
The paper introduces SWE-NFI, a benchmark that assesses how coding agents can make non-functional, behavior-preserving improvements to Python code, using real pull‑request tasks an…
To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
Zhihao Lin, Junhua Zhu, Mingyi Zhou +5
LLM-based agents for program repair are increasingly built on a "generate-run-revise" paradigm, iteratively executing tests to evaluate and refine patches. This execution-based app…
Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement
Xin Wang, Liangtai Sun, Yaoming Zhu +8
Existing code-generation benchmarks score a single mapping from a complete prompt to a one-shot output. However, real web development is different. Users seldom write a full spec a…
Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs
He Yang Yuan, Xin Wang, Kundi Yao +3
Logging code plays an important role in software systems by recording key events and behaviors, which are essential for debugging and monitoring. However, insecure logging practice…
LLM4Perf: Large Language Models Are Effective Samplers for Multi-Objective Performance Modeling
Xin Wang, Zhenhao Li, Zishuo Ding
The performance of modern software systems is critically dependent on their complex configuration options. Building accurate performance models to navigate this vast space requires…
Defects4Log: Benchmarking LLMs for Logging Code Defect Detection and Reasoning
Xin Wang, Zhenhao Li, Zishuo Ding
Logging code is written by developers to capture system runtime behavior and plays a vital role in debugging, performance analysis, and system monitoring. However, defects in loggi…