UnitedQA: A Hybrid Approach for Open Domain Question Answering
arXiv:2101.00178
Abstract
To date, most of recent work under the retrieval-reader framework for open-domain QA focuses on either extractive or generative reader exclusively. In this paper, we study a hybrid approach for leveraging the strengths of both models. We apply novel techniques to enhance both extractive and generative readers built upon recent pretrained neural language models, and find that proper training methods can provide large improvement over previous state-of-the-art models. We demonstrate that a simple hybrid approach by combining answers from both readers can efficiently take advantages of extractive and generative answer inference strategies and outperforms single models as well as homogeneous ensembles. Our approach outperforms previous state-of-the-art models by 3.3 and 2.7 points in exact match on NaturalQuestions and TriviaQA respectively.
arXiv commentACL 2021 camera-ready
References in corpus (6)
- Distilling the Knowledge in a Neural Network
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- Robust Large Margin Deep Neural Networks
- RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
- NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
- Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answering