An Understanding-Oriented Robust Machine Reading Comprehension Model

Abstract

Although existing machine reading comprehension models are making rapid progress on many datasets, they are far from robust. In this paper, we propose an understanding-oriented machine reading comprehension model to address three kinds of robustness issues, which are over sensitivity, over stability and generalization. Specifically, we first use a natural language inference module to help the model understand the accurate semantic meanings of input questions so as to address the issues of over sensitivity and over stability. Then in the machine reading comprehension module, we propose a memory-guided multi-head attention method that can further well understand the semantic meanings of input questions and passages. Third, we propose a multilanguage learning mechanism to address the issue of generalization. Finally, these modules are integrated with a multi-task learning based method. We evaluate our model on three benchmark datasets that are designed to measure models robustness, including DuReader (robust) and two SQuAD-related datasets. Extensive experiments show that our model can well address the mentioned three kinds of robustness issues. And it achieves much better results than the compared state-of-the-art models on all these datasets under different evaluation metrics, even under some extreme and unfair evaluations. The source code of our work is available at: https://github.com/neukg/RobustMRC.

arXiv commentAccepted by TALLIP

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Bidirectional Attention Flow for Machine Comprehension
• ERNIE: Enhanced Representation through Knowledge Integration
• Attention-over-Attention Neural Networks for Reading Comprehension
• Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering
• A Deep Cascade Model for Multi-Document Reading Comprehension
• Smarnet: Teaching Machines to Read and Comprehend Like Human
• Benchmarking Robustness of Machine Reading Comprehension Models