130 citations · 1k across the 78 of their papers we have counts for
79 papers
UserSimCRS v2: Simulation-Based Evaluation for Conversational Recommender Systems
Nolwenn Bernard, Krisztian Balog
Resources for simulation-based evaluation of conversational recommender systems (CRSs) are scarce. The UserSimCRS toolkit was introduced to address this gap. In this work, we prese…
Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Prediction
Andreas Konstantin Kruff, Christin Katharina Kreutz, Timo Breuer +2
Validating user simulation is a difficult task due to the lack of established measures and benchmarks, which makes it challenging to assess whether a simulator accurately reflects…
Limitations of Current Evaluation Practices for Conversational Recommender Systems and the Potential of User Simulation
Nolwenn Bernard, Krisztian Balog
Research and development on conversational recommender systems (CRSs) critically depends on sound and reliable evaluation methodologies. However, the interactive nature of these sy…
The Indispensable Role of User Simulation in the Pursuit of AGI
Krisztian Balog, ChengXiang Zhai
Progress toward Artificial General Intelligence (AGI) faces significant bottlenecks, particularly in rigorously evaluating complex interactive systems and acquiring the vast intera…
SimLab: A Platform for Simulation-based Evaluation of Conversational Information Access Systems
Nolwenn Bernard, Sharath Chandra Etagi Suresh, Krisztian Balog +1
Progress in conversational information access (CIA) systems has been hindered by the difficulty of evaluating such systems with reproducible experiments. While user simulation offe…
Second SIGIR Workshop on Simulations for Information Access (Sim4IA 2025)
Philipp Schaer, Christin Katharina Kreutz, Krisztian Balog +2
Simulations in information access (IA) have recently gained interest, as shown by various tutorials and workshops around that topic. Simulations can be key contributors to central…