6 papers
Validating Search Query Simulations: A Taxonomy of Measures
Andreas Konstantin Kruff, Nolwenn Bernard, Philipp Schaer
Assessing the validity of user simulators when used for the evaluation of information retrieval systems remains an open question, constraining their effective use and the reliabili…
Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Prediction
Andreas Konstantin Kruff, Christin Katharina Kreutz, Timo Breuer +2
Validating user simulation is a difficult task due to the lack of established measures and benchmarks, which makes it challenging to assess whether a simulator accurately reflects…
LISP -- A Rich Interaction Dataset and Loggable Interactive Search Platform
Jana Isabelle Friese, Andreas Konstantin Kruff, Philipp Schaer +2
We present a reusable dataset and accompanying infrastructure for studying human search behavior in Interactive Information Retrieval (IIR). The dataset combines detailed interacti…
Second SIGIR Workshop on Simulations for Information Access (Sim4IA 2025)
Philipp Schaer, Christin Katharina Kreutz, Krisztian Balog +2
Simulations in information access (IA) have recently gained interest, as shown by various tutorials and workshops around that topic. Simulations can be key contributors to central…
Evaluating Contrastive Feedback for Effective User Simulations
Andreas Konstantin Kruff, Timo Breuer, Philipp Schaer
The use of Large Language Models (LLMs) for simulating user behavior in the domain of Interactive Information Retrieval has recently gained significant popularity. However, their a…
Scientometric Analysis of the German IR Community within TREC & CLEF
A. K. Kruff, P. Schaer
Within this study, the influence of the German Information Retrieval community on the retrieval campaigns Text Retrieval Conference (TREC) and Conference and Labs of the Evaluation…