2 citations · 2 across the 14 of their papers we have counts for
17 papers
Beyond Task Completion: Measuring Interaction Cost in Terminal User Interfaces
Ruida Hu, Yuanhao Wang, Chao Peng +2
Large language models (LLMs) are increasingly used through terminal user interfaces (TUIs), yet task completion alone does not capture how difficult an interface is to understand a…
Can LLMs Test Terminal User Interfaces?
Chao Peng, Ruida Hu, Ajitha Rajan +3
Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated te…
TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development
Mingyu Chen, Yakun Zhang, Zihao Xie +5
In modern software development, the rapid advancement of Large Language Models (LLMs) has made the end-to-end transformation of Natural Language Requirements (NLRs) into executable…
Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
Xinchen Wang, Ruida Hu, Cuiyun Gao +2
Software documentation is crucial for repository comprehension. While Large Language Models (LLMs) advance documentation generation from code snippets to entire repositories, exist…
Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
Ruida Hu, Xinchen Wang, Chao Peng +2
The evolution of Large Language Models (LLMs) has catalyzed a paradigm shift towards intent-driven software development, where autonomous agents are expected to design and deliver…
Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice
Ruida Hu, Xinchen Wang, Xin-Cheng Wen +5
Code review is a cornerstone of software quality assurance, and recent advances in Large Language Models (LLMs) have shown promise in its automation. However, existing benchmarks f…
Benchmarking Code LLM-as-a-Judge via Constrained Perturbation and Calibration
Ruiqi Wang, Xinchen Wang, Cuiyun Gao +3
Code from LLM-assisted development may contain defects like API hallucinations, motivating the development of code evaluation metrics to assess such code before deployment. As conv…
A Fine Evaluation Method for Cube Copying Test for Early Detection of Alzheimer's Disease
Xinyu Jiang, Cuiyun Gao, Wenda Huang +8
Background: Impairment of visual spatial cognitive function is the most common early clinical manifestation of Alzheimer's Disease (AD). When the Montreal Cognitive Assessment (MoC…
VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specifications
Hao Zhu, Jia Li, Cuiyun Gao +7
Large language models (LLMs) have achieved remarkable progress in code understanding tasks. However, they demonstrate limited performance in vulnerability detection and struggle to…
Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
Trae Research Team, Pengfei Gao, Zhao Tian +12
Software issue resolution is a critical challenge in software engineering and has garnered increasing attention in recent years. With the rapid advancement of large language models…
LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models
Zhanyue Qin, Yue Ding, Deyuan Liu +7
Nowadays, Large Language Models (LLMs) have attracted widespread attention due to their powerful performance. However, due to the unavoidable exposure to socially biased data durin…
CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
Xinchen Wang, Pengfei Gao, Chao Peng +2
Large language models (LLMs) have demonstrated strong capabilities in code generation, underscoring the critical need for rigorous and comprehensive evaluation. Existing evaluation…
Repo2Run: Automated Building Executable Environment for Code Repository at Scale
Ruida Hu, Chao Peng, Xinchen Wang +2
Scaling up executable code data is significant for improving language models' software engineering capability. The intricate nature of the process makes it labor-intensive, time-co…
CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
Ruida Hu, Chao Peng, Jingyi Ren +6
In this work, we introduce CodeRepoQA, a large-scale benchmark specifically designed for evaluating repository-level question-answering capabilities in the field of software engine…
A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
Ruida Hu, Chao Peng, Jingyi Ren +6
Automatically resolving software issues is crucial for software development in practice, impacting the software quality and user experience. The process of resolving real-world iss…
AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
Xinchen Wang, Pengfei Gao, Xiangxin Meng +4
In software maintenance, bug reproduction is essential for effective fault localization and repair. Manually writing reproduction scripts is a time-consuming task with high require…
Developers' Perceptions on the Impact of ChatGPT in Software Development: A Survey
Thiago S. Vaillant, Felipe Deveza de Almeida, Paulo Anselmo M. S. Neto +3
As Large Language Models (LLMs), including ChatGPT and analogous systems, continue to advance, their robust natural language processing capabilities and diverse applications have g…