12 citations · 23 across the 8 of their papers we have counts for
7 papers · 1 filter
JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell +1
The effective training and evaluation of retrieval systems require a substantial amount of relevance judgments, which are traditionally collected from human assessors -- a process…
LLMJudge: LLMs for Relevance Judgments
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell +6
The LLMJudge challenge is organized as part of the LLM4Eval workshop at SIGIR 2024. Test collections are essential for evaluating information retrieval (IR) systems. The evaluation…
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi +6
The first edition of the workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) took place in July 2024, co-located with the ACM SIGIR Conference…
Patterns of gender-specializing query reformulation
Amifa Raj, Bhaskar Mitra, Nick Craswell +1
Users of search systems often reformulate their queries by adding query terms to reflect their evolving information need or to more precisely express their information need when th…
Taking Search to Task
Chirag Shah, Ryen W. White, Paul Thomas +3
The importance of tasks in information retrieval (IR) has been long argued for, addressed in different ways, often ignored, and frequently revisited. For decades, scholars made a c…
Are We There Yet? A Decision Framework for Replacing Term Based Retrieval with Dense Retrieval Systems
Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra +2
Recently, several dense retrieval (DR) models have demonstrated competitive performance to term-based retrieval that are ubiquitous in search systems. In contrast to term-based mat…