Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
arXiv:2005.06249
Abstract
Machine reading comprehension (MRC) aims to teach machines to read and comprehend human languages, which is a long-standing goal of natural language processing (NLP). With the burst of deep neural networks and the evolution of contextualized language models (CLMs), the research of MRC has experienced two significant breakthroughs. MRC and CLM, as a phenomenon, have a great impact on the NLP community. In this survey, we provide a comprehensive and comparative review on MRC covering overall research topics about 1) the origin and development of MRC and CLM, with a particular focus on the role of CLMs; 2) the impact of MRC and CLM to the NLP community; 3) the definition, datasets, and evaluation of MRC; 4) general MRC architecture and technical methods in the view of two-stage Encoder-Decoder solving architecture from the insights of the cognitive process of humans; 5) previous highlights, emerging topics, and our empirical analysis, among which we especially focus on what works in different periods of MRC researches. We propose a full-view categorization and new taxonomies on these topics. The primary views we have arrived at are that 1) MRC boosts the progress from language processing to understanding; 2) the rapid improvement of MRC systems greatly benefits from the development of CLMs; 3) the theme of MRC is gradually moving from shallow text matching to cognitive reasoning.
51 pages
References in corpus (23)
- Sequence to Sequence Learning with Neural Networks
- REALM: Retrieval-Augmented Language Model Pre-Training
- Machine Comprehension Using Match-LSTM and Answer Pointer
- Assessing BERT's Syntactic Abilities
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Understanding the Behaviors of BERT in Ranking
- Quasar: Datasets for Question Answering by Search and Reading
- DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
- MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
- Embracing data abundance: BookTest Dataset for Reading Comprehension
- Option Comparison Network for Multiple-choice Reading Comprehension
- Dynamically Fused Graph Network for Multi-hop Reasoning
- Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering
- Machine Reading Comprehension: a Literature Review
- A Survey on Neural Machine Reading Comprehension
- Question Answering is a Format; When is it Useful?
- Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents
- From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gap
- MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension
- Multi-style Generative Reading Comprehension
- Graph Sequential Network for Reasoning over Sequences
- Explicit Sentence Compression for Neural Machine Translation
- MATINF: A Jointly Labeled Large-Scale Dataset for Classification, Question Answering and Summarization