170 citations · 225 across the 17 of their papers we have counts for
17 papers
Retrieving Supporting Evidence for LLMs Generated Answers
Siqing Huo, Negar Arabzadeh, Charles L. A. Clarke
Current large language models (LLMs) can exhibit near-human levels of performance on many natural language tasks, including open-domain question answering. Unfortunately, they also…
Evaluating Open-Domain Question Answering in the Era of Large Language Models
Ehsan Kamalloo, Nouha Dziri, Charles L. A. Clarke +1
Lexical matching remains the de facto evaluation method for open-domain question answering (QA). Unfortunately, lexical matching fails completely when a plausible candidate answer…
Perspectives on Large Language Models for Relevance Judgment
Guglielmo Faggioli, Laura Dietz, Charles Clarke +8
When asked, large language models (LLMs) like ChatGPT claim that they can assist with relevance judgments but it is not clear whether automated judgments can reliably be used in ev…
Early Stage Sparse Retrieval with Entity Linking
Dahlia Shehata, Negar Arabzadeh, Charles L. A. Clarke
Despite the advantages of their low-resource settings, traditional sparse retrievers depend on exact matching approaches between high-dimensional bag-of-words (BoW) representations…
Unsupervised Question Clarity Prediction Through Retrieved Item Coherency
Negar Arabzadeh, Mahsa Seifikar, Charles L. A. Clarke
Despite recent progress on conversational systems, they still do not perform smoothly and coherently when faced with ambiguous requests. When questions are unclear, conversational…
Human Preferences as Dueling Bandits
Xinyi Yan, Chengxi Luo, Charles L. A. Clarke +3
The dramatic improvements in core information retrieval tasks engendered by neural rankers create a need for novel evaluation methods. If every ranker returns highly relevant items…
Translating Human Mobility Forecasting through Natural Language Generation
Hao Xue, Flora D. Salim, Yongli Ren +1
Existing human mobility forecasting models follow the standard design of the time-series prediction model which takes a series of numerical values as input to generate a numerical…
Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection
Negar Arabzadeh, Xinyi Yan, Charles L. A. Clarke
Over the last few years, contextualized pre-trained transformer models such as BERT have provided substantial improvements on information retrieval tasks. Recent approaches based o…
Shallow pooling for sparse labels
Negar Arabzadeh, Alexandra Vtyurina, Xinyi Yan +1
Recent years have seen enormous gains in core IR tasks, including document and passage ranking. Datasets and leaderboards, and in particular the MS MARCO datasets, illustrate the d…
Assessing top- preferences
Charles L. A. Clarke, Alexandra Vtyurina, Mark D. Smucker
Assessors make preference judgments faster and more consistently than graded judgments. Preference judgments can also recognize distinctions between items that appear equivalent un…
Efficient and Effective Tail Latency Minimization in Multi-Stage Retrieval Systems
Joel Mackenzie, J. Shane Culpepper, Roi Blanco +3
Scalable web search systems typically employ multi-stage retrieval architectures, where an initial stage generates a set of candidate documents that are then pruned and re-ranked.…
Ten Blue Links on Mars
Charles L. A. Clarke, Gordon V. Cormack, Jimmy Lin +1
This paper explores a simple question: How would we provide a high-quality search experience on Mars, where the fundamental physical limit is speed-of-light propagation delays on t…
Dynamic Trade-Off Prediction in Multi-Stage Retrieval Systems
J. Shane Culpepper, Charles L. A. Clarke, Jimmy Lin
Modern multi-stage retrieval systems are comprised of a candidate generation stage followed by one or more reranking stages. In such an architecture, the quality of the final ranke…
The Effects of Latency Penalties in Evaluating Push Notification Systems
Luchen Tan, Jimmy Lin, Adam Roegiest +1
We examine the effects of different latency penalties in the evaluation of push notification systems, as operationalized in the TREC 2015 Microblog track evaluation. The purpose of…
Assessing Efficiency-Effectiveness Tradeoffs in Multi-Stage Retrieval Systems Without Using Relevance Judgments
Charles L. A. Clarke, J. Shane Culpepper, Alistair Moffat
Large-scale retrieval systems are often implemented as a cascading sequence of phases -- a first filtering step, in which a large set of candidate documents are extracted using a s…
A Family of Rank Similarity Measures based on Maximized Effectiveness Difference
Luchen Tan, Clarke L. A. Clarke
Rank similarity measures provide a method for quantifying differences between search engine results without the need for relevance judgments. For example, the providers of a search…
Efficient and Effective Spam Filtering and Re-ranking for Large Web Datasets
Gordon V. Cormack, Mark D. Smucker, Charles L. A. Clarke
The TREC 2009 web ad hoc and relevance feedback tasks used a new document collection, the ClueWeb09 dataset, which was crawled from the general Web in early 2009. This dataset cont…