MIX : a Multi-task Learning Approach to Solve Open-Domain Question Answering

Abstract

This paper introduces MIX, a multi-task deep learning approach to solve open-ended question-answering. First, we design our system as a multi-stage pipeline of 3 building blocks: a BM25-based Retriever to reduce the search space, a RoBERTa-based Scorer, and an Extractor to rank retrieved paragraphs and extract relevant text spans, respectively. Eventually, we further improve the computational efficiency of our system to deal with the scalability challenge: thanks to multi-task learning, we parallelize the close tasks solved by the Scorer and the Extractor. Our system is on par with state-of-the-art performances on the squad-open benchmark while being simpler conceptually.

8 pages, 7 figures, 3 tables

References in corpus (11)

• Decoupled Weight Decay Regularization
• Bidirectional Attention Flow for Machine Comprehension
• Mixed Precision Training
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• End-to-End Open-Domain Question Answering with BERTserini
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
• R${}^3$: Reinforced Reader-Ranker for Open-Domain Question Answering
• Efficient and Robust Question Answering from Minimal Context over Documents
• Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading
• Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering
• Accelerating Real-Time Question Answering via Question Generation