activity
20242026
most citedAn Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4

12 citations · 21 across the 24 of their papers we have counts for

collaborators
Showing 2024Show all

11 papers · 1 filter

cs.CL2024★ 1 cited

LLM-based Discriminative Reasoning for Knowledge Graph Question Answering

Mufan Xu, Kehai Chen, Xuefeng Bai +3

Large language models (LLMs) based on generative pre-trained Transformer have achieved remarkable performance on knowledge graph question-answering (KGQA) tasks. However, LLMs ofte…

cs.CL2024

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

Shaoqing Zhang, Zhuosheng Zhang, Kehai Chen +4

Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechan…

cs.CL2024

PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment

Dongxu Liu, Bing Xu, Yinzhuo Chen +4

Reinforcement Learning from Human Feedback (RLHF) has been proven to be an effective method for preference alignment of large language models (LLMs) and is widely used in the post-…

cs.CL2024

LLM-based Translation Inference with Iterative Bilingual Understanding

Andong Chen, Kehai Chen, Yang Xiang +5

The remarkable understanding and generation capabilities of large language models (LLMs) have greatly improved translation performance. However, incorrect understanding of the sent…

cs.AI2024

Dynamic Planning for LLM-based Graphical User Interface Automation

Shaoqing Zhang, Zhuosheng Zhang, Kehai Chen +4

The advent of large language models (LLMs) has spurred considerable interest in advancing autonomous LLMs-based agents, particularly in intriguing applications within smartphone gr…

cs.CL2024

Mitigating the Bias of Large Language Model Evaluation

Hongli Zhou, Hui Huang, Yunfei Long +5

Recently, there has been a trend of evaluating the Large Language Model (LLM) quality in the flavor of LLM-as-a-Judge, namely leveraging another LLM to evaluate the current output…

cs.CL2024★ 1 cited

Large Language Models for Classical Chinese Poetry Translation: Benchmarking, Evaluating, and Improving

Andong Chen, Lianzhang Lou, Kehai Chen +5

Different from the traditional translation tasks, classical Chinese poetry translation requires both adequacy and fluency in translating culturally and historically significant con…

cs.CL2024

DUAL-REFLECT: Enhancing Large Language Models for Reflective Translation through Dual Learning Feedback Mechanisms

Andong Chen, Lianzhang Lou, Kehai Chen +5

Recently, large language models (LLMs) enhanced by self-reflection have achieved promising performance on machine translation. The key idea is guiding LLMs to generate translation…

cs.CL2024★ 4 cited

A Survey on Human Preference Learning for Large Language Models

Ruili Jiang, Kehai Chen, Xuefeng Bai +6

The recent surge of versatile large language models (LLMs) largely depends on aligning increasingly capable foundation models with human intentions by preference learning, enhancin…

cs.CL2024

Self-Evaluation of Large Language Model based on Glass-box Features

Hui Huang, Yingqi Qu, Jing Liu +4

The proliferation of open-source Large Language Models (LLMs) underscores the pressing need for evaluation methods. Existing works primarily rely on external evaluators, focusing o…

cs.CL2024★ 12 cited

An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4

Hui Huang, Xingyuan Bu, Hongli Zhou +5

Recently, there has been a growing trend of utilizing Large Language Model (LLM) to evaluate the quality of other LLMs. Many studies have fine-tuned judge models based on open-sour…