12 citations · 12 across the 6 of their papers we have counts for
7 papers · 1 filter
LegalRewardBench: Benchmarking Reward Models for Legal Preference Evaluation
Hui Huang, Jie Zhou, Hongli Zhou +7
Reward models are central to preference alignment for legal tasks, yet existing reward model benchmarks are still primarily centered on general-domain preferences. In this work, we…
Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization
Hongli Zhou, Hui Huang, Rui Zhang +5
Large language model (LLM)-based judges are widely adopted for automated evaluation and reward modeling, yet their judgments are often affected by judgment biases. Accurately evalu…
RM-Distiller: Exploiting Generative LLM for Reward Model Distillation
Hongli Zhou, Hui Huang, Wei Liu +8
Reward models (RMs) play a pivotal role in aligning large language models (LLMs) with human preferences. Due to the difficulty of obtaining high-quality human preference annotation…
Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theory
Hongli Zhou, Hui Huang, Ziqing Zhao +10
The evaluation of large language models (LLMs) via benchmarks is widespread, yet inconsistencies between different leaderboards and poor separability among top models raise concern…
Think-J: Learning to Think for Generative LLM-as-a-Judge
Hui Huang, Yancheng He, Hongli Zhou +5
LLM-as-a-Judge refers to the automatic modeling of preferences for responses generated by Large Language Models (LLMs), which is of significant importance for both LLM evaluation a…
Mitigating the Bias of Large Language Model Evaluation
Hongli Zhou, Hui Huang, Yunfei Long +5
Recently, there has been a trend of evaluating the Large Language Model (LLM) quality in the flavor of LLM-as-a-Judge, namely leveraging another LLM to evaluate the current output…
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
Hui Huang, Xingyuan Bu, Hongli Zhou +5
Recently, there has been a growing trend of utilizing Large Language Model (LLM) to evaluate the quality of other LLMs. Many studies have fine-tuned judge models based on open-sour…