6 papers · 1 filter
SWR-Bench: Assessing LLM Performance in Real-World Code Review Comment Generation
Zhengran Zeng, Ruikai Shi, Keke Han +7
Automated Code Review (ACR) is crucial for software quality, yet existing benchmarks often fail to reflect real-world complexities, hindering the evaluation of modern Large Languag…
GALA: Multimodal Graph Alignment for Bug Localization in Automated Program Repair
Zhuoyao Liu, Zhengran Zeng, Shu-Dong Huang +3
Large Language Model (LLM)-based Automated Program Repair (APR) has shown strong potential on textual benchmarks, yet struggles in multimodal scenarios where bugs are reported with…
An Empirical Study on Influence-Based Pretraining Data Selection for Code Large Language Models
Chengli Xing, Zhengran Zeng, Gexiang Fang +3
Recent advancements in code large language models (Code-LLMs) have demonstrated remarkable capabilities in resolving programming related tasks. Meanwhile, researchers have recogniz…
Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
Zhengran Zeng, Yixin Li, Rui Xie +2
The development of LLM-based autonomous agents for end-to-end software development represents a significant paradigm shift in software engineering. However, the scientific evaluati…
Seed&Steer: Guiding Large Language Models with Compilable Prefix and Branch Signals for Unit Test Generation
Shuaiyu Zhou, Zhengran Zeng, Xiaoling Zhou +3
Unit tests play a vital role in the software development lifecycle. Recent advances in Large Language Model (LLM)-based approaches have significantly improved automated test genera…
A Survey on Evaluating Large Language Models in Code Generation Tasks
Liguo Chen, Qi Guo, Hongrui Jia +9
This paper provides a comprehensive review of the current methods and metrics used to evaluate the performance of Large Language Models (LLMs) in code generation tasks. With the ra…