4 papers
LiveRAG: A diverse Q&A dataset with varying difficulty level for RAG evaluation
David Carmel, Simone Filice, Guy Horowitz +4
With Retrieval Augmented Generation (RAG) becoming more and more prominent in generative AI solutions, there is an emerging need for systematically evaluating their effectiveness.…
Do RAG Systems Really Suffer From Positional Bias?
Florin Cuconasu, Simone Filice, Guy Horowitz +2
Retrieval Augmented Generation enhances LLM accuracy by adding passages retrieved from an external corpus to the LLM prompt. This paper investigates how positional bias - the tende…
SIGIR 2025 -- LiveRAG Challenge Report
David Carmel, Simone Filice, Guy Horowitz +6
The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants…
Generating Diverse Q&A Benchmarks for RAG Evaluation with DataMorgana
Simone Filice, Guy Horowitz, David Carmel +3
Evaluating Retrieval-Augmented Generation (RAG) systems, especially in domain-specific contexts, requires benchmarks that address the distinctive requirements of the applicative sc…