Embedding-based Zero-shot Retrieval through Query Generation

Abstract

Passage retrieval addresses the problem of locating relevant passages, usually from a large corpus, given a query. In practice, lexical term-matching algorithms like BM25 are popular choices for retrieval owing to their efficiency. However, term-based matching algorithms often miss relevant passages that have no lexical overlap with the query and cannot be finetuned to downstream datasets. In this work, we consider the embedding-based two-tower architecture as our neural retrieval model. Since labeled data can be scarce and because neural retrieval models require vast amounts of data to train, we propose a novel method for generating synthetic training data for retrieval. Our system produces remarkable results, significantly outperforming BM25 on 5 out of 6 datasets tested, by an average of 2.45 points for Recall@1. In some cases, our model trained on synthetic data can even outperform the same model trained on real data

References in corpus (10)

• Universal Sentence Encoder
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• REALM: Retrieval-Augmented Language Model Pre-Training
• Passage Re-ranking with BERT
• Question Answering and Question Generation as Dual Tasks
• Unsupervised Question Answering by Cloze Translation
• Pre-training Tasks for Embedding-based Large-scale Retrieval
• Universal Text Representation from BERT: An Empirical Study
• Document Ranking with a Pretrained Sequence-to-Sequence Model
• Passage Ranking with Weak Supervision

Cited by in corpus (4)

• Information Retrieval: Recent Advances and Beyond
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• Contrastive Document Representation Learning with Graph Attention Networks