12 citations · 19 across the 10 of their papers we have counts for
12 papers · 1 filter
SuiteEval: Simplifying Retrieval Benchmarks
Andrew Parry, Debasis Ganguly, Sean MacAvaney
Information retrieval evaluation often suffers from fragmented practices -- varying dataset subsets, aggregation methods, and pipeline configurations -- that undermine reproducibil…
Disentangling Locality and Entropy in Ranking Distillation
Andrew Parry, Debasis Ganguly, Sean MacAvaney
The training process of ranking models involves two key data selection decisions: a sampling strategy, and a labeling strategy. Modern ranking systems, especially those for perform…
Modeling Ranking Properties with In-Context Learning
Nilanjan Sinhababu, Andrew Parry, Debasis Ganguly +1
While standard IR models are primarily designed to optimize relevance, real-world search often needs to balance additional objectives such as diversity and fairness. These objectiv…
Variations in Relevance Judgments and the Shelf Life of Test Collections
Andrew Parry, Maik Fröbe, Harrisen Scells +5
The fundamental property of Cranfield-style evaluations, that system rankings are stable even when assessors disagree on individual relevance decisions, was validated on traditiona…
MechIR: A Mechanistic Interpretability Framework for Information Retrieval
Andrew Parry, Catherine Chen, Carsten Eickhoff +1
Mechanistic interpretability is an emerging diagnostic approach for neural models that has gained traction in broader natural language processing domains. This paradigm aims to pro…
Training on the Test Model: Contamination in Ranking Distillation
Vishakha Suresh Kalal, Andrew Parry, Sean MacAvaney
Neural approaches to ranking based on pre-trained language models are highly effective in ad-hoc search. However, the computational expense of these models can limit their applicat…