54 citations · 162 across the 15 of their papers we have counts for
13 papers · 1 filter
Formalized Information Needs Improve Large-Language-Model Relevance Judgments
Jüri Keller, Maik Fröbe, Björn Engelmann +4
Cranfield-style retrieval evaluations with too few or too many relevant documents or with low inter-assessor agreement on relevance can reduce the reliability of observations. In e…
Multilingual and Domain-Agnostic Tip-of-the-Tongue Query Generation for Simulated Evaluation
Xuhong He, To Eun Kim, Maik Fröbe +3
Tip-of-the-Tongue (ToT) retrieval benchmarks have largely focused on English, limiting their applicability to multilingual information access. In this work, we construct multilingu…
Variations in Relevance Judgments and the Shelf Life of Test Collections
Andrew Parry, Maik Fröbe, Harrisen Scells +5
The fundamental property of Cranfield-style evaluations, that system rankings are stable even when assessors disagree on individual relevance decisions, was validated on traditiona…
Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data. However, distilled models do…
Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders
Ferdinand Schlatt, Maik Fröbe, Harrisen Scells +6
Existing cross-encoder models can be categorized as pointwise, pairwise, or listwise. Pairwise and listwise models allow passage interactions, which typically makes them more effec…
Evaluating Generative Ad Hoc Information Retrieval
Lukas Gienapp, Harrisen Scells, Niklas Deckers +9
Recent advances in large language models have enabled the development of viable generative retrieval systems. Instead of a traditional document ranking, generative retrieval system…
The Information Retrieval Experiment Platform
Maik Fröbe, Jan Heinrich Reimer, Sean MacAvaney +6
We integrate ir_datasets, ir_measures, and PyTerrier with TIRA in the Information Retrieval Experiment Platform (TIREx) to promote more standardized, reproducible, scalable, and ev…
The Archive Query Log: Mining Millions of Search Result Pages of Hundreds of Search Engines from 25 Years of Web Archives
Jan Heinrich Reimer, Sebastian Schmidt, Maik Fröbe +5
The Archive Query Log (AQL) is a previously unused, comprehensive query log collected at the Internet Archive over the last 25 years. Its first version includes 356 million queries…
The Infinite Index: Information Retrieval on Generative Text-To-Image Models
Niklas Deckers, Maik Fröbe, Johannes Kiesel +4
Conditional generative models such as DALL-E and Stable Diffusion generate images based on a user-defined text, the prompt. Finding and refining prompts that produce a desired imag…
Sparse Pairwise Re-ranking with Pre-trained Transformers
Lukas Gienapp, Maik Fröbe, Matthias Hagen +1
Pairwise re-ranking models predict which of two documents is more relevant to a query and then aggregate a final ranking from such preferences. This is often more effective than po…
How Train-Test Leakage Affects Zero-shot Retrieval
Maik Fröbe, Christopher Akiki, Martin Potthast +1
Neural retrieval models are often trained on (subsets of) the millions of queries of the MS MARCO / ORCAS datasets and then tested on the 250 Robust04 queries or other TREC benchma…
SCAI-QReCC Shared Task on Conversational Question Answering
Svitlana Vakulenko, Johannes Kiesel, Maik Fröbe
Search-Oriented Conversational AI (SCAI) is an established venue that regularly puts a spotlight upon the recent work advancing the field of conversational search. SCAI'21 was orga…
The Impact of Main Content Extraction on Near-Duplicate Detection
Maik Fröbe, Matthias Hagen, Janek Bevendorff +6
Commercial web search engines employ near-duplicate detection to ensure that users see each relevant result only once, albeit the underlying web crawls typically include (near-)dup…