9 papers · 1 filter
SQLite is Enough. Lexical, Semantic, and Hybrid Search with scrydb
Timo Breuer
This work introduces scrydb, a Python library that enables lexical, semantic, and hybrid search within SQLite. For lexical search, scrydb leverages SQLite's full-text search extens…
Formalized Information Needs Improve Large-Language-Model Relevance Judgments
Jüri Keller, Maik Fröbe, Björn Engelmann +4
Cranfield-style retrieval evaluations with too few or too many relevant documents or with low inter-assessor agreement on relevance can reduce the reliability of observations. In e…
Sim4IA-Bench: A User Simulation Benchmark Suite for Next Query and Utterance Prediction
Andreas Konstantin Kruff, Christin Katharina Kreutz, Timo Breuer +2
Validating user simulation is a difficult task due to the lack of established measures and benchmarks, which makes it challenging to assess whether a simulator accurately reflects…
Second SIGIR Workshop on Simulations for Information Access (Sim4IA 2025)
Philipp Schaer, Christin Katharina Kreutz, Krisztian Balog +2
Simulations in information access (IA) have recently gained interest, as shown by various tutorials and workshops around that topic. Simulations can be key contributors to central…
Evaluating Contrastive Feedback for Effective User Simulations
Andreas Konstantin Kruff, Timo Breuer, Philipp Schaer
The use of Large Language Models (LLMs) for simulating user behavior in the domain of Interactive Information Retrieval has recently gained significant popularity. However, their a…
Report on the Workshop on Simulations for Information Access (Sim4IA 2024) at SIGIR 2024
Timo Breuer, Christin Katharina Kreutz, Norbert Fuhr +15
This paper is a report of the Workshop on Simulations for Information Access (Sim4IA) workshop at SIGIR 2024. The workshop had two keynotes, a panel discussion, nine lightning talk…