108 citations · 293 across the 20 of their papers we have counts for
4 papers · 1 filter
Can We Hide Machines in the Crowd? Quantifying Equivalence in LLM-in-the-loop Annotation Tasks
Jiaman He, Zikang Leng, Dana McKay +2
Many evaluations of large language models (LLMs) in text annotation focus primarily on the correctness of the output, typically comparing model-generated labels to human-annotated…
Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE): Manifesto
Christine Bauer, Li Chen, Nicola Ferro +19
During the workshop, we deeply discussed what CONversational Information ACcess (CONIAC) is and its unique features, proposing a world model abstracting it, and defined the Convers…
Characterising Topic Familiarity and Query Specificity Using Eye-Tracking Data
Jiaman He, Zikang Leng, Dana McKay +2
Eye-tracking data has been shown to correlate with a user's knowledge level and query formulation behaviour. While previous work has focused primarily on eye gaze fixations for att…
Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment
Julian A. Schnabel, Johanne R. Trippas, Falk Scholer +1
The effectiveness of search systems is evaluated using relevance labels that indicate the usefulness of documents for specific queries and users. While obtaining these relevance la…