From the 1 of 5 linked papers with an AI index.
5 papers
ExplainBench: Evaluating Code Explanations from Agents
Zhiyuan Pan, Sungmin Kang, Imam Nur Bani Yusuf +1
The paper introduces ExplainBench, a benchmark that automatically evaluates how trustworthy the explanations generated by code‑writing LLM agents are, by checking if the explanatio…
Code Review Agent Benchmark
Yuntong Zhang, Zhiyuan Pan, Imam Nur Bani Yusuf +3
Software engineering agents have shown significant promise in writing code. As AI agents permeate code writing, and generate huge volumes of code automatically -- the matter of cod…
CATCODER: Repository-Level Code Generation with Relevant Code and Type Context
Zhiyuan Pan, Xing Hu, Xin Xia +1
Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, repository-level code generation presents unique challenges, particularly…
Open the Oyster: Empirical Evaluation and Improvement of Code Reasoning Confidence in LLMs
Shufan Wang, Xing Hu, Junkai Chen +2
With the widespread application of large language models (LLMs) in the field of code intelligence, increasing attention has been paid to the reliability and controllability of thei…
Re-Evaluating Code LLM Benchmarks Under Semantic Mutation
Zhiyuan Pan, Xing Hu, Xin Xia +1
In the era of large language models (LLMs), code benchmarks have become an important research area in software engineering and are widely used by practitioners. These benchmarks ev…