4 papers · 1 filter
How Fine-Grained Should a RAG Benchmark Be? A Hierarchical Framework for Synthetic Question Generation
Chase M. Fensore, Kaustubh Dhole, Jason Fan +2
Evaluating retrieval-augmented generation (RAG) systems requires benchmarks that capture diverse question characteristics, yet practitioners lack empirical guidance on which dimens…
Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge
Chase Fensore, Kaustubh Dhole, Joyce C Ho +1
We present our submission to the LiveRAG Challenge 2025, which evaluates retrieval-augmented generation (RAG) systems on dynamic test sets using the FineWeb-10BT corpus. Our final…
ConQRet: Benchmarking Fine-Grained Evaluation of Retrieval Augmented Argumentation with LLM Judges
Kaustubh D. Dhole, Kai Shu, Eugene Agichtein
Computational argumentation, which involves generating answers or summaries for controversial topics like abortion bans and vaccination, has become increasingly important in today'…
Leveraging Interesting Facts to Enhance User Engagement with Conversational Interfaces
Nikhita Vedula, Giuseppe Castellucci, Eugene Agichtein +2
Conversational Task Assistants (CTAs) guide users in performing a multitude of activities, such as making recipes. However, ensuring that interactions remain engaging, interesting,…