54 citations · 72 across the 9 of their papers we have counts for
5 papers · 1 filter
Benchmarking Code LLM-as-a-Judge via Constrained Perturbation and Calibration
Ruiqi Wang, Xinchen Wang, Cuiyun Gao +3
Code from LLM-assisted development may contain defects like API hallucinations, motivating the development of code evaluation metrics to assess such code before deployment. As conv…
An Empirical Study of Knowledge Distillation for Code Understanding Tasks
Ruiqi Wang, Zezhou Yang, Cuiyun Gao +2
Pre-trained language models (PLMs) have emerged as powerful tools for code understanding. However, deploying these PLMs in large-scale applications faces practical challenges due t…
Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering
Ruiqi Wang, Jiyu Guo, Cuiyun Gao +3
Recently, large language models (LLMs) have been deployed to tackle various software engineering (SE) tasks like code generation, significantly advancing the automation of SE tasks…
The Current Challenges of Software Engineering in the Era of Large Language Models
Cuiyun Gao, Xing Hu, Shan Gao +2
With the advent of large language models (LLMs) in the artificial intelligence (AI) area, the field of software engineering (SE) has also witnessed a paradigm shift. These models,…
ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code
Jia Feng, Jiachen Liu, Cuiyun Gao +4
In recent years, the application of large language models (LLMs) to code-related tasks has gained significant attention. However, existing evaluation benchmarks often focus on limi…