MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
arXiv:2005.02507
Abstract
Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus (Ahmad et al.,2019).This paper presents MultiReQA, anew multi-domain ReQA evaluation suite com-posed of eight retrieval QA tasks drawn from publicly available QA datasets. We provide the first systematic retrieval based evaluation over these datasets using two supervised neural models, based on fine-tuning BERT andUSE-QA models respectively, as well as a surprisingly strong information retrieval baseline,BM25. Five of these tasks contain both train-ing and test data, while three contain test data only. Performance on the five tasks with train-ing data shows that while a general model covering all domains is achievable, the best performance is often obtained by training exclusively on in-domain data.
References in corpus (5)
- SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
- Pre-training Tasks for Embedding-based Large-scale Retrieval
- End-to-End Retrieval in Continuous Space
- Multilingual Universal Sentence Encoder for Semantic Retrieval
- LAReQA: Language-agnostic answer retrieval from a multilingual pool
Cited by in corpus (3)
- QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
- Learning to Reuse Distractors to support Multiple Choice Question Generation in Education
- Zero-Shot Estimation of Base Models' Weights in Ensemble of Machine Reading Comprehension Systems for Robust Generalization