Overview of the TREC 2020 deep learning track

Abstract

This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a passage retrieval task, each with hundreds of thousands of human-labeled training queries. We evaluate using single-shot TREC-style evaluation, to give us a picture of which ranking methods work best when large data is available, with much more comprehensive relevance labeling on the small number of test queries. This year we have further evidence that rankers with BERT-style pretraining outperform other rankers in the large data regime.

arXiv commentarXiv admin note: substantial text overlap with arXiv:2003.07820

Cited by in corpus (27)

• Information Retrieval: Recent Advances and Beyond
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching
• The Information Retrieval Experiment Platform
• Generative Relevance Feedback with Large Language Models
• Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction
• Lexically-Accelerated Dense Retrieval
• CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typos
• How Discriminative Are Your Qrels? How To Study the Statistical Significance of Document Adjudication Methods
• Towards a Better Understanding Human Reading Comprehension with Brain Signals
• To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
• Optimizing Guided Traversal for Fast Learned Sparse Retrieval
• The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
• SpaDE: Improving Sparse Representations using a Dual Document Encoder for First-stage Retrieval
• SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes
• Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval
• CODER: An efficient framework for improving retrieval through COntextual Document Embedding Reranking
• On the Interpolation of Contextualized Term-based Ranking with BM25 for Query-by-Example Retrieval
• Soft Prompt Decoding for Multilingual Dense Retrieval
• Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach
• LexBoost: Improving Lexical Document Retrieval with Nearest Neighbors
• Human Preferences as Dueling Bandits
• Building Efficient and Effective OpenQA Systems for Low-Resource Languages
• Hybrid and Collaborative Passage Reranking
• Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls
• Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
• Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback