Predicting Relevance based on Assessor Disagreement: Analysis and Practical Applications for Search Evaluation

Thomas Demeester, Robin Aly, Djoerd Hiemstra +2

Evaluation of search engines relies on assessments of search results for selected test queries, from which we would ideally like to draw conclusions in terms of relevance of the re…