5 papers
DoGMaTiQ: Automated Generation of Question-and-Answer Nuggets for Report Evaluation
Bryan Li, William Walden, Yu Hou +6
Evaluation of long-form, citation-backed reports has lately received significant attention due to the wide-scale adoption of retrieval-augmented generation (RAG) systems. Core to m…
Search for Coverage: Learning Coverage-Aware Retrieval with Augmented Sub-Question Answerability
Jia-Huei Ju, Eugene Yang, Trevor Adriaanse +2
Long-form Retrieval-Augmented Generation (RAG) brings the challenge of coverage-based ranking, because ranking methods must ensure the inclusion of comprehensive relevant nuggets (…
ICICLE: Expanding Retrieval with In-Context Documents
Yu-Chen Den, Yung-Yu Shih, Zhi Rui Tam +4
Generative retrieval (GR) maps queries directly to document identifiers (docids) using parametric knowledge, However, this design makes corpus expansion costly: adding new document…
Insider Knowledge: How Much Can RAG Systems Gain from Evaluation Secrets?
Laura Dietz, Bryan Li, Eugene Yang +3
RAG systems are increasingly evaluated and optimized using LLM judges, an approach that is rapidly becoming the dominant paradigm for system assessment. Nugget-based approaches in…
Incorporating Q&A Nuggets into Retrieval-Augmented Generation
Laura Dietz, Bryan Li, Gabrielle Liu +5
RAGE systems integrate ideas from automatic evaluation (E) into Retrieval-augmented Generation (RAG). As one such example, we present Crucible, a Nugget-Augmented Generation System…