32 citations · 122 across the 14 of their papers we have counts for
7 papers · 1 filter
Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR
Nandan Thakur, Luiz Bonifacio, Maik Fröbe +5
The zero-shot effectiveness of neural retrieval models is often evaluated on the BEIR benchmark -- a combination of different IR evaluation datasets. Interestingly, previous studie…
LLMs Can Patch Up Missing Relevance Judgments in Evaluation
Shivani Upadhyay, Ehsan Kamalloo, Jimmy Lin
Unjudged documents or holes in information retrieval benchmarks are considered non-relevant in evaluation, yielding no gains in measuring effectiveness. However, these missing judg…
Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard
Ehsan Kamalloo, Nandan Thakur, Carlos Lassance +3
BEIR is a benchmark dataset for zero-shot evaluation of information retrieval models across 18 different domain/task combinations. In recent years, we have witnessed the growing po…
Evaluating Embedding APIs for Information Retrieval
Ehsan Kamalloo, Xinyu Zhang, Odunayo Ogundepo +4
The ever-increasing size of language models curtails their widespread availability to the community, thereby galvanizing many companies into offering access to large language model…
Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval
Jimmy Lin, David Alfonso-Hermelo, Vitor Jeronymo +8
The advent of multilingual language models has generated a resurgence of interest in cross-lingual information retrieval (CLIR), which is the task of searching documents in one lan…
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages
Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo +6
MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual dataset we have built for the WSDM 2023 Cup challenge that focuses on ad hoc retrieval…
A Coherent Unsupervised Model for Toponym Resolution
Ehsan Kamalloo, Davood Rafiei
Toponym Resolution, the task of assigning a location mention in a document to a geographic referent (i.e., latitude/longitude), plays a pivotal role in analyzing location-aware con…