6 papers
CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational Search
Nolwenn Bernard, Jüri Keller, Philipp Schaer
This paper presents the participation of the Cologne Information Retrieval group in the iKAT SCAI 2026 shared task. We use an agentic conversational search system, equipped with to…
UserSimCRS v2: Simulation-Based Evaluation for Conversational Recommender Systems
Nolwenn Bernard, Krisztian Balog
Resources for simulation-based evaluation of conversational recommender systems (CRSs) are scarce. The UserSimCRS toolkit was introduced to address this gap. In this work, we prese…
Validating Search Query Simulations: A Taxonomy of Measures
Andreas Konstantin Kruff, Nolwenn Bernard, Philipp Schaer
Assessing the validity of user simulators when used for the evaluation of information retrieval systems remains an open question, constraining their effective use and the reliabili…
SimLab: A Platform for Simulation-based Evaluation of Conversational Information Access Systems
Nolwenn Bernard, Sharath Chandra Etagi Suresh, Krisztian Balog +1
Progress in conversational information access (CIA) systems has been hindered by the difficulty of evaluating such systems with reproducible experiments. While user simulation offe…
Limitations of Current Evaluation Practices for Conversational Recommender Systems and the Potential of User Simulation
Nolwenn Bernard, Krisztian Balog
Research and development on conversational recommender systems (CRSs) critically depends on sound and reliable evaluation methodologies. However, the interactive nature of these sy…
CRS Arena: Crowdsourced Benchmarking of Conversational Recommender Systems
Nolwenn Bernard, Hideaki Joko, Faegheh Hasibi +1
We introduce CRS Arena, a research platform for scalable benchmarking of Conversational Recommender Systems (CRS) based on human feedback. The platform displays pairwise battles be…