Showing cs.IRShow all
3 papers · 1 filter
cs.IR2023
New Metrics to Encourage Innovation and Diversity in Information Retrieval Approaches
Mehmet Deniz Türkmen, Matthew Lease, Mucahid Kutlu
In evaluation campaigns, participants often explore variations of popular, state-of-the-art baselines as a low-risk strategy to achieve competitive results. While effective, this c…
cs.IR2018
Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably
Mucahid Kutlu, Tyler McDonnell, Aashish Sheshadri +2
Crowdsourcing offers an affordable and scalable means to collect relevance judgments for IR test collections. However, crowd assessors may show higher variance in judgment quality…
cs.IR2017
EveTAR: Building a Large-Scale Multi-Task Test Collection over Arabic Tweets
Maram Hasanain, Reem Suwaileh, Tamer Elsayed +2
This article introduces a new language-independent approach for creating a large-scale high-quality test collection of tweets that supports multiple information retrieval (IR) task…