Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation

Abstract

A major obstacle to the wide-spread adoption of neural retrieval models is that they require large supervised training sets to surpass traditional term-based techniques, which are constructed from raw corpora. In this paper, we propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap. The question generation system is trained on general domain data, but is applied to documents in the targeted domain. This allows us to create arbitrarily large, yet noisy, question-passage relevance pairs that are domain specific. Furthermore, when this is coupled with a simple hybrid term-neural model, first-stage retrieval performance can be improved further. Empirically, we show that this is an effective strategy for building neural passage retrieval models in the absence of large training corpora. Depending on the domain, this technique can even approach the accuracy of supervised models.

arXiv comment14 pages, 4 figures

References in corpus (14)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• Don't Decay the Learning Rate, Increase the Batch Size
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• End-to-End Open-Domain Question Answering with BERTserini
• Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
• Passage Re-ranking with BERT
• Document Expansion by Query Prediction
• Simple Applications of BERT for Ad Hoc Document Retrieval
• Pre-training Tasks for Embedding-based Large-scale Retrieval
• Neural Vector Spaces for Unsupervised Information Retrieval
• Sparse, Dense, and Attentional Representations for Text Retrieval
• End-to-End Retrieval in Continuous Space
• Efficient Inner Product Approximation in Hybrid Spaces