55 citations · 81 across the 7 of their papers we have counts for
8 papers
Does Pass Rate Tell the Whole Story? Evaluating Design Constraint Compliance in LLM-based Issue Resolution
Kai Yu, Zhenhao Zhou, Junhao Zeng +8
Repository-level issue resolution benchmarks have become a standard testbed for evaluating LLM-based agents, yet success is still predominantly measured by test pass rates. In prac…
Project-Level C-to-Rust Translation via Pointer Knowledge Graphs
Zhiqiang Yuan, Wenjun Mao, Zhuo Chen +4
Translating C code into safe Rust is an effective way to ensure memory safety. Compared to rule-based approaches, which often produce largely unsafe Rust code, LLM-based methods ge…
TransAgent: Enhancing LLM-Based Code Translation via Fine-Grained Execution Alignment
Zhiqiang Yuan, Weitong Chen, Hanlin Wang +3
Code translation transforms code between programming languages while preserving functionality, which is critical in software development and maintenance. While traditional learning…
Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation
Zhiqiang Yuan, Junwei Liu, Qiancheng Zi +3
In this work, we evaluate 10 open-source instructed LLMs on four representative code comprehension and generation tasks. We have the following main findings. First, for the zero-sh…
No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation
Zhiqiang Yuan, Yiling Lou, Mingwei Liu +4
Unit testing is essential in detecting bugs in functionally-discrete program units. Manually writing high-quality unit tests is time-consuming and laborious. Although traditional t…
SE Factual Knowledge in Frozen Giant Code Model: A Study on FQN and its Retrieval
Qing Huang, Dianshu Liao, Zhenchang Xing +4
Pre-trained giant code models (PCMs) start coming into the developers' daily practices. Understanding what types of and how much software knowledge is packed into PCMs is the found…