25 citations · 41 across the 6 of their papers we have counts for
6 papers
CoverageBench: Evaluating Information Coverage across Tasks and Domains
Saron Samuel, Andrew Yates, Dawn Lawrie +4
We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Inf…
Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage
Saron Samuel, Alexander Martin, Eugene Yang +5
Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation. While the relat…
QueryBuilder: Human-in-the-Loop Query Development for Information Retrieval
Hemanth Kandula, Damianos Karakos, Haoling Qiu +4
Frequently, users of an Information Retrieval (IR) system start with an overarching information need (a.k.a., an analytic task) and proceed to define finer-grained queries covering…
Don't Use LLMs to Make Relevance Judgments
Ian Soboroff
Making the relevance judgments for a TREC-style test collection can be complex and expensive. A typical TREC track usually involves a team of six contractors working for 2-4 weeks.…
Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performance
Somnath Banerjee, Avik Halder, Rajarshi Mandal +4
The integration of pretrained language models (PLMs) like BERT and GPT has revolutionized NLP, particularly for English, but it has also created linguistic imbalances. This paper s…
On the Evaluation of Machine-Generated Reports
James Mayfield, Eugene Yang, Dawn Lawrie +10
Large Language Models (LLMs) have enabled new ways to satisfy information needs. Although great strides have been made in applying them to settings like document ranking and short-…