2 citations · 4 across the 14 of their papers we have counts for
6 papers · 1 filter
CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering
Ruida Hu, Chao Peng, Jingyi Ren +6
In this work, we introduce CodeRepoQA, a large-scale benchmark specifically designed for evaluating repository-level question-answering capabilities in the field of software engine…
A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
Ruida Hu, Chao Peng, Jingyi Ren +6
Automatically resolving software issues is crucial for software development in practice, impacting the software quality and user experience. The process of resolving real-world iss…
AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
Xinchen Wang, Pengfei Gao, Xiangxin Meng +4
In software maintenance, bug reproduction is essential for effective fault localization and repair. Manually writing reproduction scripts is a time-consuming task with high require…
RepoMasterEval: Evaluating Code Completion via Real-World Repositories
Qinyun Wu, Chao Peng, Pengfei Gao +9
With the growing reliance on automated code completion tools in software development, the need for comprehensive evaluation benchmarks has become critical. Existing benchmarks focu…
VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection
Xin-Cheng Wen, Xinchen Wang, Yujia Chen +3
Deep Learning (DL)-based methods have proven to be effective for software vulnerability detection, with a potential for substantial productivity enhancements for detecting vulnerab…
ReposVul: A Repository-Level High-Quality Vulnerability Dataset
Xinchen Wang, Ruida Hu, Cuiyun Gao +3
Open-Source Software (OSS) vulnerabilities bring great challenges to the software security and pose potential risks to our society. Enormous efforts have been devoted into automate…