Making Neural Machine Reading Comprehension Faster
arXiv:1904.00796
Abstract
This study aims at solving the Machine Reading Comprehension problem where questions have to be answered given a context passage. The challenge is to develop a computationally faster model which will have improved inference time. State of the art in many natural language understanding tasks, BERT model, has been used and knowledge distillation method has been applied to train two smaller models. The developed models are compared with other models which have been developed with the same intention.
References in corpus (24)
- Adam: A Method for Stochastic Optimization
- Distilling the Knowledge in a Neural Network
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Layer Normalization
- Machine Comprehension Using Match-LSTM and Answer Pointer
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Xception: Deep Learning with Depthwise Separable Convolutions
- ReasoNet: Learning to Stop Reading in Machine Comprehension
- Learning Recurrent Span Representations for Extractive Question Answering
- Multi-Perspective Context Matching for Machine Comprehension
- DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
- MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
- Learning to Compute Word Embeddings On the Fly
- Words or Characters? Fine-grained Gating for Reading Comprehension
- Stochastic Answer Networks for Machine Reading Comprehension
- Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
- Phase Conductor on Multi-layered Attentions for Machine Comprehension
- Structural Embedding of Syntactic Trees for Machine Comprehension
- Smarnet: Teaching Machines to Read and Comprehend Like Human
- Ruminating Reader: Reasoning with Gated Multi-Hop Attention
- Neural Machine Translation for Bilingually Scarce Scenarios: A Deep Multi-task Learning Approach
- Lightweight Convolutional Approaches to Reading Comprehension on SQuAD
- DTMT: A Novel Deep Transition Architecture for Neural Machine Translation
- Syntax-based Attention Model for Natural Language Inference