Unsupervised Domain Adaptation on Reading Comprehension

Abstract

Reading comprehension (RC) has been studied in a variety of datasets with the boosted performance brought by deep neural networks. However, the generalization capability of these models across different domains remains unclear. To alleviate this issue, we are going to investigate unsupervised domain adaptation on RC, wherein a model is trained on labeled source domain and to be applied to the target domain with only unlabeled samples. We first show that even with the powerful BERT contextual representation, the performance is still unsatisfactory when the model trained on one dataset is directly applied to another target dataset. To solve this, we provide a novel conditional adversarial self-training method (CASe). Specifically, our approach leverages a BERT model fine-tuned on the source dataset along with the confidence filtering to generate reliable pseudo-labeled samples in the target domain for self-training. On the other hand, it further reduces domain distribution discrepancy through conditional adversarial learning across domains. Extensive experiments show our approach achieves comparable accuracy to supervised models on multiple large-scale benchmark datasets.

arXiv comment8 pages, 6 figures, 5 tables, Accepted by AAAI 2020

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Teaching Machines to Read and Comprehend
• Conditional Adversarial Domain Adaptation
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• Multi-Task Deep Neural Networks for Natural Language Understanding
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• NewsQA: A Machine Comprehension Dataset
• Question Answering through Transfer Learning from Large Fine-grained Supervision Data
• Supervised and Unsupervised Transfer Learning for Question Answering