2 citations · 10 across the 31 of their papers we have counts for
17 papers · 1 filter
SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA
Jing Zhang, Lianghong Guo, Yanlin Wang +7
The Development Knowledge Question Answering (Dev Knowledge QA) task aims to provide natural language answers to knowledge-seeking questions during software development. To investi…
Knowledge-Graph-Driven Data Synthesis for Low-Resource Software Development: A HarmonyOS Case Study
Mingwei Liu, Zheng Pei, Yanlin Wang +5
In low-resource framework development (e.g., HarmonyOS), large language models (LLMs) often lack sufficient pre-training exposure, resulting in poor code generation performance. Al…
Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach
Yuanheng Li, Zhuoyang Chen, Xiaoyun Liu +5
As large language models (LLMs) become increasingly capable, concerns over the unauthorized use of copyrighted and licensed content in their training data have grown, especially in…
Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation
Anji Li, Mingwei Liu, Zhenxi Chen +5
Automated unit test generation using large language models (LLMs) holds great promise but often struggles with generating tests that are both correct and maintainable in real-world…
EffiReasonTrans: RL-Optimized Reasoning for Code Translation
Yanlin Wang, Rongyi Ou, Yanli Wang +6
Code translation is a crucial task in software development and maintenance. While recent advancements in large language models (LLMs) have improved automated code translation accur…
Generating High-Quality Datasets for Code Editing via Open-Source Language Models
Zekai Zhang, Mingwei Liu, Zhenxi Chen +7
Code editing plays a vital role in software engineering, requiring developers to adjust existing code according to natural language instructions while keeping functionality intact…
EvolMathEval: Towards Evolvable Benchmarks for Mathematical Reasoning via Evolutionary Testing
Shengbo Wang, Mingwei Liu, Zike Li +4
The rapid advancement of Large Language Models (LLMs) poses a significant challenge to existing mathematical reasoning benchmarks. However, these benchmarks tend to become easier o…
A Hierarchical and Evolvable Benchmark for Fine-Grained Code Instruction Following with Multi-Turn Feedback
Guoliang Duan, Mingwei Liu, Yanlin Wang +3
Large language models (LLMs) have advanced significantly in code generation, yet their ability to follow complex programming instructions with layered and diverse constraints remai…
AutoEvoEval: An Automated Framework for Evolving Close-Ended LLM Evaluation Data
JiaRu Wu, Mingwei Liu
Large language models (LLMs) have shown remarkable performance on various tasks, but existing evaluation benchmarks are often static and insufficient to fully assess their robustne…
AdaDec: A Uncertainty-Guided Lookahead Decoding Framework for LLM-Based Code Generation
Kaifeng He, Mingwei Liu, Chong Wang +4
Code generation with large language models (LLMs) is highly sensitive to token selection during decoding, particularly at uncertain decision points that influence program logic. Wh…
Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation
Mingwei Liu, Juntao Li, Ying Wang +9
Despite recent advances in Large Language Models (LLMs) for code generation, the quality of LLM-generated code still faces significant challenges. One significant issue is code rep…
FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
Dekun Dai, MingWei Liu, Anji Li +5
Code repair is a fundamental task in software development, facilitating efficient bug resolution and software maintenance. Although large language models (LLMs) have demonstrated c…
What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond
Wenchao Gu, Juntao Chen, Yanlin Wang +6
Repository-level code generation remains challenging due to complex code dependencies and the limitations of large language models (LLMs) in processing long contexts. While retriev…
RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
Linxi Liang, Jing Gong, Mingwei Liu +5
Large Language Models (LLMs) have become pivotal tools for automating code generation in software development. However, these models face significant challenges in producing versio…
A Preliminary Study on the Robustness of Code Generation by Large Language Models
Zike Li, Mingwei Liu, Anji Li +4
Robustness is a critical factor for reliable code generation by large language models, yet most evaluations focus on correctness and overlook key issues such as missing input valid…
Evolving Triple Knowledge-Augmented LLMs for Code Translation in Repository Context
Guangsheng Ou, Mingwei Liu, Yuxuan Chen +5
Large language models (LLMs) have behaved well in function-level code translation without repository-level context. However, the performance of LLMs in repository-level context cod…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…