2 citations · 3 across the 7 of their papers we have counts for
7 papers
New Metrics to Encourage Innovation and Diversity in Information Retrieval Approaches
Mehmet Deniz Türkmen, Matthew Lease, Mucahid Kutlu
In evaluation campaigns, participants often explore variations of popular, state-of-the-art baselines as a low-risk strategy to achieve competitive results. While effective, this c…
Patent Search Using Triplet Networks Based Fine-Tuned SciBERT
Utku Umur Acikalin, Mucahid Kutlu
In this paper, we propose a novel method for the prior-art search task. We fine-tune SciBERT transformer model using Triplet Network approach, allowing us to represent each patent…
An Information Retrieval Approach to Building Datasets for Hate Speech Detection
Md Mustafizur Rahman, Dinesh Balakrishnan, Dhiraj Murthy +2
Building a benchmark dataset for hate speech detection presents various challenges. Firstly, because hate speech is relatively rare, random sampling of tweets to annotate is very i…
Understanding and Predicting Characteristics of Test Collections in Information Retrieval
Md Mustafizur Rahman, Mucahid Kutlu, Matthew Lease
Research community evaluations in information retrieval, such as NIST's Text REtrieval Conference (TREC), build reusable test collections by pooling document rankings submitted by…
Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably
Mucahid Kutlu, Tyler McDonnell, Aashish Sheshadri +2
Crowdsourcing offers an affordable and scalable means to collect relevance judgments for IR test collections. However, crowd assessors may show higher variance in judgment quality…
EveTAR: Building a Large-Scale Multi-Task Test Collection over Arabic Tweets
Maram Hasanain, Reem Suwaileh, Tamer Elsayed +2
This article introduces a new language-independent approach for creating a large-scale high-quality test collection of tweets that supports multiple information retrieval (IR) task…
Intelligent Topic Selection for Low-Cost Information Retrieval Evaluation: A New Perspective on Deep vs. Shallow Judging
Mucahid Kutlu, Tamer Elsayed, Matthew Lease
While test collections provide the cornerstone for Cranfield-based evaluation of information retrieval (IR) systems, it has become practically infeasible to rely on traditional poo…