2 citations · 4 across the 14 of their papers we have counts for
16 papers
Beyond Task Completion: Measuring Interaction Cost in Terminal User Interfaces
Ruida Hu, Yuanhao Wang, Chao Peng +2
Large language models (LLMs) are increasingly used through terminal user interfaces (TUIs), yet task completion alone does not capture how difficult an interface is to understand a…
Can LLMs Test Terminal User Interfaces?
Chao Peng, Ruida Hu, Ajitha Rajan +3
Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated te…
Evaluating Repository-level Software Documentation via Question Answering and Feature-Driven Development
Xinchen Wang, Ruida Hu, Cuiyun Gao +2
Software documentation is crucial for repository comprehension. While Large Language Models (LLMs) advance documentation generation from code snippets to entire repositories, exist…
Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
Ruida Hu, Xinchen Wang, Chao Peng +2
The evolution of Large Language Models (LLMs) has catalyzed a paradigm shift towards intent-driven software development, where autonomous agents are expected to design and deliver…
Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice
Ruida Hu, Xinchen Wang, Xin-Cheng Wen +5
Code review is a cornerstone of software quality assurance, and recent advances in Large Language Models (LLMs) have shown promise in its automation. However, existing benchmarks f…
SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement
Zexun Zhan, Shuzheng Gao, Ruida Hu +1
Large language models (LLMs) have achieved remarkable progress in code generation. However, existing benchmarks mainly formalize the task as a static, single-turn problem, overlook…
Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling
Trae Research Team, Pengfei Gao, Zhao Tian +12
Software issue resolution is a critical challenge in software engineering and has garnered increasing attention in recent years. With the rapid advancement of large language models…
CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation
Xinchen Wang, Pengfei Gao, Chao Peng +2
Large language models (LLMs) have demonstrated strong capabilities in code generation, underscoring the critical need for rigorous and comprehensive evaluation. Existing evaluation…
Repo2Run: Automated Building Executable Environment for Code Repository at Scale
Ruida Hu, Chao Peng, Xinchen Wang +2
Scaling up executable code data is significant for improving language models' software engineering capability. The intricate nature of the process makes it labor-intensive, time-co…
CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
Ruida Hu, Chao Peng, Jingyi Ren +6
In this work, we introduce CodeRepoQA, a large-scale benchmark specifically designed for evaluating repository-level question-answering capabilities in the field of software engine…
A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
Ruida Hu, Chao Peng, Jingyi Ren +6
Automatically resolving software issues is crucial for software development in practice, impacting the software quality and user experience. The process of resolving real-world iss…
AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
Xinchen Wang, Pengfei Gao, Xiangxin Meng +4
In software maintenance, bug reproduction is essential for effective fault localization and repair. Manually writing reproduction scripts is a time-consuming task with high require…
RepoMasterEval: Evaluating Code Completion via Real-World Repositories
Qinyun Wu, Chao Peng, Pengfei Gao +9
With the growing reliance on automated code completion tools in software development, the need for comprehensive evaluation benchmarks has become critical. Existing benchmarks focu…
VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection
Xin-Cheng Wen, Xinchen Wang, Yujia Chen +3
Deep Learning (DL)-based methods have proven to be effective for software vulnerability detection, with a potential for substantial productivity enhancements for detecting vulnerab…
ReposVul: A Repository-Level High-Quality Vulnerability Dataset
Xinchen Wang, Ruida Hu, Cuiyun Gao +3
Open-Source Software (OSS) vulnerabilities bring great challenges to the software security and pose potential risks to our society. Enormous efforts have been devoted into automate…
Less is More? An Empirical Study on Configuration Issues in Python PyPI Ecosystem
Yun Peng, Ruida Hu, Ruoke Wang +3
Python is widely used in the open-source community, largely owing to the extensive support from diverse third-party libraries within the PyPI ecosystem. Nevertheless, the utilizati…