3 papers
cs.IR2026
A Sensitivity-Aware Test Collection for Search Among Personal Information
Jack McKechnie, Graham McDonald, Craig Macdonald
Traditional search tasks aim to satisfy user information needs by returning a subset of a collection of documents, ranked by the documents' relevance to a user query. However, some…
cs.CY2026
All Eyes on the Ranker: Participatory Auditing to Surface Blind Spots in Ranked Search Results
Anna Marie Rezk, Patrizia Di Campli San Vito, Ayah Soufan +3
Search engines that present users with a ranked list of search results are a fundamental technology for providing public access to information. Evaluations of such systems are typi…
cs.IR2025
Measuring Hypothesis Testing Errors in the Evaluation of Retrieval Systems
Jack McKechnie, Graham McDonald, Craig Macdonald
The evaluation of Information Retrieval (IR) systems typically uses query-document pairs with corresponding human-labelled relevance assessments (qrels). These qrels are used to de…