Publications (14)
Constructing and Evaluating Declarative RAG Pipelines in PyTerrier
Craig Macdonald, Jinyuan Fang, Andrew Parry +1
Search engines often follow a pipeline architecture, where complex but effective reranking components are used to refine the results of an initial retrieval. Retrieval augmented ge…
Disentangling Locality and Entropy in Ranking Distillation
Andrew Parry, Debasis Ganguly, Sean MacAvaney
The training process of ranking models involves two key data selection decisions: a sampling strategy, and a labeling strategy. Modern ranking systems, especially those for perform…
SuiteEval: Simplifying Retrieval Benchmarks
Andrew Parry, Debasis Ganguly, Sean MacAvaney
Information retrieval evaluation often suffers from fragmented practices -- varying dataset subsets, aggregation methods, and pipeline configurations -- that undermine reproducibil…
Top-Down Partitioning for Efficient List-Wise Ranking
Andrew Parry, Sean MacAvaney, Debasis Ganguly
Large Language Models (LLMs) have significantly impacted many facets of natural language processing and information retrieval. Unlike previous encoder-based approaches, the enlarge…
Variations in Relevance Judgments and the Shelf Life of Test Collections
Andrew Parry, Maik Fröbe, Harrisen Scells +5
The fundamental property of Cranfield-style evaluations, that system rankings are stable even when assessors disagree on individual relevance decisions, was validated on traditiona…
MechIR: A Mechanistic Interpretability Framework for Information Retrieval
Andrew Parry, Catherine Chen, Carsten Eickhoff +1
Mechanistic interpretability is an emerging diagnostic approach for neural models that has gained traction in broader natural language processing domains. This paradigm aims to pro…