Multilingual Question Answering from Formatted Text applied to Conversational Agents
arXiv:1910.04659
Abstract
Recent advances with language models (e.g. BERT, XLNet, ...), have allowed surpassing human performance on complex NLP tasks such as Reading Comprehension. However, labeled datasets for training are available mostly in English which makes it difficult to acknowledge progress in other languages. Fortunately, models are now pre-trained on unlabeled data from hundreds of languages and exhibit interesting transfer abilities from one language to another. In this paper, we show that multilingual BERT is naturally capable of zero-shot transfer for an extractive Question Answering task (eQA) from English to other languages. More specifically, it outperforms the best previously known baseline for transfer to Japanese and French. Moreover, using a recently published large eQA French dataset, we are able to further show that (1) zero-shot transfer provides results really close to a direct training on the target language and (2) combination of transfer and training on target is the best option overall. We finally present a practical application: a multilingual conversational agent called Kate which answers to HR-related questions in several languages directly from the content of intranet pages.
References in corpus (13)
- Distributed Representations of Words and Phrases and their Compositionality
- Distributed Representations of Sentences and Documents
- Cross-lingual Language Model Pretraining
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- An efficient framework for learning sentence representations
- XNLI: Evaluating Cross-lingual Sentence Representations
- How multilingual is Multilingual BERT?
- RACE: Large-scale ReAding Comprehension Dataset From Examinations
- Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
- Multilingual Extractive Reading Comprehension by Runtime Machine Translation
- Zero-Resource Translation with Multi-Lingual Neural Machine Translation
- FQuAD: French Question Answering Dataset
- Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian