4 papers
LLM-Assisted Pseudo-Relevance Feedback
David Otero, Javier Parapar
Query expansion is a long-standing technique to mitigate vocabulary mismatch in ad hoc Information Retrieval. Pseudo-relevance feedback methods, such as RM3, estimate an expanded q…
Can LLMs Evaluate What They Cannot Annotate? Revisiting LLM Reliability in Hate Speech Detection
Paloma Piot, David Otero, Patricia Martín-Rodilla +1
Hate speech spreads widely online, harming individuals and communities, making automatic detection essential for large-scale moderation, yet detecting it remains difficult. Part of…
Towards Reliable Testing for Multiple Information Retrieval System Comparisons
David Otero, Javier Parapar, Álvaro Barreiro
Null Hypothesis Significance Testing is the \textit{de facto} tool for assessing effectiveness differences between Information Retrieval systems. Researchers use statistical tests…
Limitations of Automatic Relevance Assessments with Large Language Models for Fair and Reliable Retrieval Evaluation
David Otero, Javier Parapar, Álvaro Barreiro
Offline evaluation of search systems depends on test collections. These benchmarks provide the researchers with a corpus of documents, topics and relevance judgements indicating wh…