4 papers · 1 filter
CoverageBench: Evaluating Information Coverage across Tasks and Domains
Saron Samuel, Andrew Yates, Dawn Lawrie +4
We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Inf…
Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage
Saron Samuel, Alexander Martin, Eugene Yang +5
Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation. While the relat…
QueryBuilder: Human-in-the-Loop Query Development for Information Retrieval
Hemanth Kandula, Damianos Karakos, Haoling Qiu +4
Frequently, users of an Information Retrieval (IR) system start with an overarching information need (a.k.a., an analytic task) and proceed to define finer-grained queries covering…
Don't Use LLMs to Make Relevance Judgments
Ian Soboroff
Making the relevance judgments for a TREC-style test collection can be complex and expensive. A typical TREC track usually involves a team of six contractors working for 2-4 weeks.…