3 citations · 4 across the 6 of their papers we have counts for
6 papers
Overview of the NTCIR-18 Automatic Evaluation of LLMs (AEOLLM) Task
Junjie Chen, Haitao Li, Zhumin Chu +2
In this paper, we provide an overview of the NTCIR-18 Automatic Evaluation of LLMs (AEOLLM) task. As large language models (LLMs) grow popular in both academia and industry, how to…
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges
Haitao Li, Junjie Chen, Qingyao Ai +4
The use of large language models (LLMs) as automated evaluation tools to assess the quality of generated natural language, known as LLMs-as-Judges, has demonstrated promising capab…
Auto-PRE: An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation
Junjie Chen, Weihang Su, Zhumin Chu +9
The rapid development of large language models (LLMs) has highlighted the need for efficient and reliable methods to evaluate their performance. Traditional evaluation methods ofte…
PRE: A Peer Review Based Large Language Model Evaluator
Zhumin Chu, Qingyao Ai, Yiteng Tu +2
The impressive performance of large language models (LLMs) has attracted considerable attention from the academic and industrial communities. Besides how to construct and train LLM…
Corrected Evaluation Results of the NTCIR WWW-2, WWW-3, and WWW-4 English Subtasks
Tetsuya Sakai, Sijie Tao, Maria Maistro +7
Unfortunately, the official English (sub)task results reported in the NTCIR-14 WWW-2, NTCIR-15 WWW-3, and NTCIR-16 WWW-4 overview papers are incorrect due to noise in the official…
Towards Better Understanding of User Satisfaction in Open-Domain Conversational Search
Zhumin Chu, Qingyao Ai, Zhihong Wang +5
With the increasing popularity of conversational search, how to evaluate the performance of conversational search systems has become an important question in the IR community. Exis…