8 papers
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
Xin Zhou, Chun Yong Chong, Kisub Kim +11
Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs. Yet existing repository-level…
Assessing and Advancing Benchmarks for Evaluating Large Language Models in Software Engineering Tasks
Xing Hu, Feifei Niu, Junkai Chen +5
Large language models (LLMs) are gaining increasing popularity in software engineering (SE) due to their unprecedented performance across various applications. These models are inc…
Human-Aligned Code Readability Assessment with Large Language Models
Wendkûuni C. Ouédraogo, Yinghua Li, Xueqi Dang +6
Code readability is crucial for software comprehension and maintenance, yet difficult to assess at scale. Traditional static metrics often fail to capture the subjective, context-s…
Beyond Surface Similarity: Evaluating LLM-Based Test Refactorings with Structural and Semantic Awareness
Wendkûuni C. Ouédraogo, Yinghua Li, Xueqi Dang +5
Large Language Models (LLMs) are increasingly used to refactor unit tests, improving readability and structure while preserving behavior. Evaluating such refactorings, however, rem…
An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks
Xin Zhou, Kisub Kim, Ting Zhang +6
Large Language Models (LLMs) and other automated techniques have been increasingly used to support software developers by generating software artifacts such as code snippets, patch…
VisDocSketcher: Towards Scalable Visual Documentation with Agentic Systems
LuÃs F. Gomes, Xin Zhou, David Lo +1
Visual documentation is an effective tool for reducing the cognitive barrier developers face when understanding unfamiliar code, enabling more intuitive comprehension. Compared to…