Towards Domain Adaptation from Limited Data for Question Answering Using Deep Neural Networks

Abstract

This paper explores domain adaptation for enabling question answering (QA) systems to answer questions posed against documents in new specialized domains. Current QA systems using deep neural network (DNN) technology have proven effective for answering general purpose factoid-style questions. However, current general purpose DNN models tend to be ineffective for use in new specialized domains. This paper explores the effectiveness of transfer learning techniques for this problem. In experiments on question answering in the automobile manual domain we demonstrate that standard DNN transfer learning techniques work surprisingly well in adapting DNN models to a new domain using limited amounts of annotated training data in the new domain.

References in corpus (7)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Bidirectional Attention Flow for Machine Comprehension
• Frustratingly Easy Domain Adaptation
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering

Cited by in corpus (4)

• Fine-tuning Strategies for Domain Specific Question Answering under Low Annotation Budget Constraints
• CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering
• Contrastive Domain Adaptation for Question Answering using Limited Text Corpora
• On the Universality of Deep Contextual Language Models