204 citations · 271 across the 9 of their papers we have counts for
10 papers
LiveRAG: A diverse Q&A dataset with varying difficulty level for RAG evaluation
David Carmel, Simone Filice, Guy Horowitz +4
With Retrieval Augmented Generation (RAG) becoming more and more prominent in generative AI solutions, there is an emerging need for systematically evaluating their effectiveness.…
Redefining Retrieval Evaluation in the Era of LLMs
Giovanni Trappolini, Florin Cuconasu, Simone Filice +2
Traditional Information Retrieval (IR) metrics, such as nDCG, MAP, and MRR, assume that human users sequentially examine documents with diminishing attention to lower ranks. This a…
SIGIR 2025 -- LiveRAG Challenge Report
David Carmel, Simone Filice, Guy Horowitz +6
The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants…
Do RAG Systems Really Suffer From Positional Bias?
Florin Cuconasu, Simone Filice, Guy Horowitz +2
Retrieval Augmented Generation enhances LLM accuracy by adding passages retrieved from an external corpus to the LLM prompt. This paper investigates how positional bias - the tende…
Generating Diverse Q&A Benchmarks for RAG Evaluation with DataMorgana
Simone Filice, Guy Horowitz, David Carmel +3
Evaluating Retrieval-Augmented Generation (RAG) systems, especially in domain-specific contexts, requires benchmarks that address the distinctive requirements of the applicative sc…
The Power of Noise: Redefining Retrieval for RAG Systems
Florin Cuconasu, Giovanni Trappolini, Federico Siciliano +5
Retrieval-Augmented Generation (RAG) has recently emerged as a method to extend beyond the pre-trained knowledge of Large Language Models by augmenting the original prompt with rel…
Alexa, Let's Work Together: Introducing the First Alexa Prize TaskBot Challenge on Conversational Task Assistance
Anna Gottardi, Osman Ipek, Giuseppe Castellucci +27
Since its inception in 2016, the Alexa Prize program has enabled hundreds of university students to explore and compete to develop conversational agents through the SocialBot Grand…
"Alexa, what do you do for fun?" Characterizing playful requests with virtual assistants
Chen Shani, Alexander Libov, Sofia Tolmach +3
Virtual assistants such as Amazon's Alexa, Apple's Siri, Google Home, and Microsoft's Cortana, are becoming ubiquitous in our daily lives and successfully help users in various dai…
Leveraging Crowdsourcing Data For Deep Active Learning - An Application: Learning Intents in Alexa
Jie Yang, Thomas Drake, Andreas Damianou +1
This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesia…
How Many Folders Do You Really Need?
Mihajlo Grbovic, Guy Halawi, Zohar Karnin +1
Email classification is still a mostly manual task. Consequently, most Web mail users never define a single folder. Recently however, automatic classification offering the same cat…