Leveraging Advantages of Interactive and Non-Interactive Models for Vector-Based Cross-Lingual Information Retrieval
arXiv:2111.01992
Abstract
Interactive and non-interactive model are the two de-facto standard frameworks in vector-based cross-lingual information retrieval (V-CLIR), which embed queries and documents in synchronous and asynchronous fashions, respectively. From the retrieval accuracy and computational efficiency perspectives, each model has its own superiority and shortcoming. In this paper, we propose a novel framework to leverage the advantages of these two paradigms. Concretely, we introduce semi-interactive mechanism, which builds our model upon non-interactive architecture but encodes each document together with its associated multilingual queries. Accordingly, cross-lingual features can be better learned like an interactive model. Besides, we further transfer knowledge from a well-trained interactive model to ours by reusing its word embeddings and adopting knowledge distillation. Our model is initialized from a multilingual pre-trained language model M-BERT, and evaluated on two open-resource CLIR datasets derived from Wikipedia and an in-house dataset collected from a real-world search engine. Extensive analyses reveal that our methods significantly boost the retrieval accuracy while maintaining the computational efficiency.
References in corpus (13)
- Distributed Representations of Words and Phrases and their Compositionality
- Distilling the Knowledge in a Neural Network
- Do Deep Nets Really Need to be Deep?
- A Deep Relevance Matching Model for Ad-hoc Retrieval
- End-to-End Neural Ad-hoc Ranking with Kernel Pooling
- Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
- Learned in Translation: Contextualized Word Vectors
- Whitening Sentence Representations for Better Semantics and Faster Retrieval
- Representation Degeneration Problem in Training Natural Language Generation Models
- Distilling Knowledge from Reader to Retriever for Question Answering
- Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval
- Exploiting Neural Query Translation into Cross Lingual Information Retrieval
- Robust Cross-lingual Embeddings from Parallel Sentences