2 papers
cs.IR2022
Relevance Assessments for Web Search Evaluation: Should We Randomise or Prioritise the Pooled Documents? (CORRECTED VERSION)
Tetsuya Sakai, Sijie Tao, Zhaohao Zeng
In the context of depth- pooling for constructing web search test collections, we compare two approaches to ordering pooled documents for relevance assessors: the prioritisation…
cs.CL2021
DCH-2: A Parallel Customer-Helpdesk Dialogue Corpus with Distributions of Annotators' Labels
Zhaohao Zeng, Tetsuya Sakai
We introduce a data set called DCH-2, which contains 4,390 real customer-helpdesk dialogues in Chinese and their English translations. DCH-2 also contains dialogue-level annotation…