117 citations · 160 across the 6 of their papers we have counts for
7 papers · 1 filter
Human Preferences as Dueling Bandits
Xinyi Yan, Chengxi Luo, Charles L. A. Clarke +3
The dramatic improvements in core information retrieval tasks engendered by neural rankers create a need for novel evaluation methods. If every ranker returns highly relevant items…
Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?
Ellen M. Voorhees, Ian Soboroff, Jimmy Lin
Neural retrieval models are generally regarded as fundamentally different from the retrieval techniques used in the late 1990's when the TREC ad hoc test collections were construct…
Searching for Scientific Evidence in a Pandemic: An Overview of TREC-COVID
Kirk Roberts, Tasmeer Alam, Steven Bedrick +6
We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COV…
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +3
The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available. Results so far indicate that…
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +1
This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a p…
TREC-COVID: Constructing a Pandemic Information Retrieval Test Collection
Ellen Voorhees, Tasmeer Alam, Steven Bedrick +6
TREC-COVID is a community evaluation designed to build a test collection that captures the information needs of biomedical researchers using the scientific literature during a pand…