Dawn J. Lawrie

ColBERTSaR: Sparsified ColBERT Index via Product Quantization

Eugene Yang, Andrew Yates, Dawn Lawrie +3

While ColBERT is an effective neural retrieval architecture, it requires a heavy index structure to support candidate set retrieval based on approximated token embeddings, gatherin…

CoverageBench: Evaluating Information Coverage across Tasks and Domains

Saron Samuel, Andrew Yates, Dawn Lawrie +4

We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Inf…

Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage

Saron Samuel, Alexander Martin, Eugene Yang +5

Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation. While the relat…

NeuCLIRTech: Chinese Monolingual and Cross-Language Information Retrieval Evaluation in a Challenging Domain

Dawn Lawrie, James Mayfield, Eugene Yang +6

Measuring advances in retrieval requires test collections with relevance judgments that can faithfully distinguish systems. This paper presents NeuCLIRTech, an evaluation collectio…

Overview of the TREC 2025 RAGTIME Track

Dawn Lawrie, Sean MacAvaney, James Mayfield +3

The principal goal of the RAG TREC Instrument for Multilingual Evaluation (RAGTIME) track at TREC is to study report generation from multilingual source documents. The track has cr…

RoutIR: Fast Serving of Retrieval Pipelines for Retrieval-Augmented Generation

Eugene Yang, Andrew Yates, Dawn Lawrie +2

Retrieval models are key components of Retrieval-Augmented Generation (RAG) systems, which generate search queries, process the documents returned, and generate a response. RAG sys…

Insider Knowledge: How Much Can RAG Systems Gain from Evaluation Secrets?

Laura Dietz, Bryan Li, Eugene Yang +3

RAG systems are increasingly evaluated and optimized using LLM judges, an approach that is rapidly becoming the dominant paradigm for system assessment. Nugget-based approaches in…

Incorporating Q&A Nuggets into Retrieval-Augmented Generation

Laura Dietz, Bryan Li, Gabrielle Liu +5

RAGE systems integrate ideas from automatic evaluation (E) into Retrieval-augmented Generation (RAG). As one such example, we present Crucible, a Nugget-Augmented Generation System…

NeuCLIRBench: A Modern Evaluation Collection for Monolingual, Cross-Language, and Multilingual Information Retrieval

Dawn Lawrie, James Mayfield, Eugene Yang +6

To measure advances in retrieval, test collections with relevance judgments that can faithfully distinguish systems are required. This paper presents NeuCLIRBench, an evaluation co…

HLTCOE at TREC 2024 NeuCLIR Track

Eugene Yang, Dawn Lawrie, Orion Weller +1

The HLTCOE team applied PLAID, an mT5 reranker, GPT-4 reranker, score fusion, and document translation to the TREC 2024 NeuCLIR track. For PLAID we included a variety of models and…

Overview of the TREC 2024 NeuCLIR Track

Dawn Lawrie, Sean MacAvaney, James Mayfield +4

The principal goal of the TREC Neural Cross-Language Information Retrieval (NeuCLIR) track is to study the effect of neural approaches on cross-language information access. The tra…

HLTCOE at LiveRAG: GPT-Researcher using ColBERT retrieval

Kevin Duh, Eugene Yang, Orion Weller +2

The HLTCOE LiveRAG submission utilized the GPT-researcher framework for researching the context of the question, filtering the returned results, and generating the final answer. Th…

Rank-K: Test-Time Reasoning for Listwise Reranking

Eugene Yang, Andrew Yates, Kathryn Ricci +4

Retrieve-and-rerank is a popular retrieval pipeline because of its ability to make slow but effective rerankers efficient enough at query time by reducing the number of comparisons…

MURR: Model Updating with Regularized Replay for Searching a Document Stream

Eugene Yang, Nicola Tonellotto, Dawn Lawrie +4

The Internet produces a continuous stream of new documents and user-generated queries. These naturally change over time based on events in the world and the evolution of language.…

Rank1: Test-Time Compute for Reranking in Information Retrieval

Orion Weller, Kathryn Ricci, Eugene Yang +3

We introduce Rank1, the first reranking model trained to take advantage of test-time compute. Rank1 demonstrates the applicability within retrieval of using a reasoning language mo…