7 papers · 1 filter
CoverageBench: Evaluating Information Coverage across Tasks and Domains
Saron Samuel, Andrew Yates, Dawn Lawrie +4
We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Inf…
Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankers
Saron Samuel, Benjamin Van Durme, Eugene Yang
While reasoning rerankers, such as Rank1, have demonstrated strong abilities in improving ranking relevance, it is unclear how they perform on other retrieval qualities such as fai…
NeuCLIRTech: Chinese Monolingual and Cross-Language Information Retrieval Evaluation in a Challenging Domain
Dawn Lawrie, James Mayfield, Eugene Yang +6
Measuring advances in retrieval requires test collections with relevance judgments that can faithfully distinguish systems. This paper presents NeuCLIRTech, an evaluation collectio…
LANCER: LLM Reranking for Nugget Coverage
Jia-Huei Ju, François G. Landry, Eugene Yang +2
Unlike short-form retrieval-augmented generation (RAG), such as factoid question answering, long-form RAG requires retrieval to provide documents covering a wide range of relevant…
RoutIR: Fast Serving of Retrieval Pipelines for Retrieval-Augmented Generation
Eugene Yang, Andrew Yates, Dawn Lawrie +2
Retrieval models are key components of Retrieval-Augmented Generation (RAG) systems, which generate search queries, process the documents returned, and generate a response. RAG sys…
NeuCLIRBench: A Modern Evaluation Collection for Monolingual, Cross-Language, and Multilingual Information Retrieval
Dawn Lawrie, James Mayfield, Eugene Yang +6
To measure advances in retrieval, test collections with relevance judgments that can faithfully distinguish systems are required. This paper presents NeuCLIRBench, an evaluation co…