1 citations · 1 across the 7 of their papers we have counts for
9 papers
DVD: A Robust Method for Detecting Variant Contamination in Large Language Model Evaluation
Renzhao Liang, Jingru Chen, Bo Jia +7
Evaluating large language models (LLMs) is increasingly confounded by \emph{variant contamination}: the training corpus contains semantically equivalent yet lexically or syntactica…
Multilingual Multimodal Software Developer for Code Generation
Linzheng Chai, Jian Yang, Shukai Liu +12
The rapid advancement of Large Language Models (LLMs) has significantly improved code generation, yet most models remain text-only, neglecting crucial visual aids like diagrams and…
SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models
Xianfu Cheng, Wei Zhang, Shiwei Zhang +16
The increasing application of multi-modal large language models (MLLMs) across various sectors have spotlighted the essence of their output reliability and accuracy, particularly t…
ExecRepoBench: Multi-level Executable Code Completion Evaluation
Jian Yang, Jiajun Zhang, Jiaxi Yang +9
Code completion has become an essential tool for daily software development. Existing evaluation benchmarks often employ static methods that do not fully capture the dynamic nature…
Evaluating and Aligning CodeLLMs on Human Preference
Jian Yang, Jiaxi Yang, Ke Jin +7
Code large language models (codeLLMs) have made significant strides in code generation. Most previous code-related benchmarks, which consist of various programming exercises along…
MdEval: Massively Multilingual Code Debugging
Shukai Liu, Linzheng Chai, Jian Yang +15
Code large language models (LLMs) have made significant progress in code debugging by directly generating the correct code based on the buggy code snippet. Programming benchmarks,…
M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluation
Jiaheng Liu, Ken Deng, Congnan Liu +13
Repository-level code completion has drawn great attention in software engineering, and several benchmark datasets have been introduced. However, existing repository-level code com…
Prometheus Chatbot: Knowledge Graph Collaborative Large Language Model for Computer Components Recommendation
Yunsheng Wang, Songhao Chen, Kevin Jin
Knowledge graphs (KGs) are essential in applications such as network alignment, question-answering, and recommender systems (RSs) since they offer structured relational data that f…
McEval: Massively Multilingual Code Evaluation
Linzheng Chai, Shukai Liu, Jian Yang +15
Code large language models (LLMs) have shown remarkable advances in code understanding, completion, and generation tasks. Programming benchmarks, comprised of a selection of code c…