117 citations · 367 across the 21 of their papers we have counts for
11 papers · 1 filter
Overview of the TREC 2022 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +4
This is the fourth year of the TREC Deep Learning track. As in previous years, we leverage the MS MARCO datasets that made hundreds of thousands of human annotated training labels…
Overview of the TREC 2021 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +2
This is the third year of the TREC Deep Learning track. As in previous years, we leverage the MS MARCO datasets that made hundreds of thousands of human annotated training labels a…
Noise-Robust Dense Retrieval via Contrastive Alignment Post Training
Daniel Campos, ChengXiang Zhai, Alessandro Magnani
The success of contextual word representations and advances in neural information retrieval have made dense vector-based retrieval a standard approach for passage and document rank…
Dense Sparse Retrieval: Using Sparse Language Models for Inference Efficient Dense Retrieval
Daniel Campos, ChengXiang Zhai
Vector-based retrieval systems have become a common staple for academic and industrial search applications because they provide a simple and scalable way of extending the search to…
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +2
Evaluation efforts such as TREC, CLEF, NTCIR and FIRE, alongside public leaderboard such as MS MARCO, are intended to encourage research and track our progress, addressing big ques…
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +3
The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available. Results so far indicate that…
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell +2
Leaderboards are a ubiquitous part of modern research in applied machine learning. By design, they sort entries into some linear order, where the top-scoring entry is recognized as…
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +1
This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a p…
ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
Nick Craswell, Daniel Campos, Bhaskar Mitra +2
Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval. However, click logs have not been…
On the Reliability of Test Collections for Evaluating Systems of Different Types
Emine Yilmaz, Nick Craswell, Bhaskar Mitra +1
As deep learning based models are increasingly being used for information retrieval (IR), a major challenge is to ensure the availability of test collections for measuring their qu…
Overview of the TREC 2019 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +2
The Deep Learning Track is a new track for TREC 2019, with the goal of studying ad hoc ranking in a large data regime. It is the first track with large human-labeled training sets,…