Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning

Abstract

While billions of non-English speaking users rely on search engines every day, the problem of ad-hoc information retrieval is rarely studied for non-English languages. This is primarily due to a lack of data set that are suitable to train ranking algorithms. In this paper, we tackle the lack of data by leveraging pre-trained multilingual language models to transfer a retrieval system trained on English collections to non-English queries and documents. Our model is evaluated in a zero-shot setting, meaning that we use them to predict relevance scores for query-document pairs in languages never seen during training. Our results show that the proposed approach can significantly outperform unsupervised retrieval techniques for Arabic, Chinese Mandarin, and Spanish. We also show that augmenting the English training collection with some examples from the target language can sometimes improve performance.

arXiv commentECIR 2020 (short)

References in corpus (8)

• Cross-lingual Language Model Pretraining
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• Deeper Text Understanding for IR with Contextual Neural Language Modeling
• Word Translation Without Parallel Data
• CEDR: Contextualized Embeddings for Document Ranking
• Simple Applications of BERT for Ad Hoc Document Retrieval
• Cross-Lingual Relevance Transfer for Document Retrieval

Cited by in corpus (4)

• Pretrained Transformers for Text Ranking: BERT and Beyond
• C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval
• Soft Prompt Decoding for Multilingual Dense Retrieval
• Cross-Lingual Training with Dense Retrieval for Document Retrieval