Attention-over-Attention Neural Networks for Reading Comprehension

Abstract

Cloze-style queries are representative problems in reading comprehension. Over the past few months, we have seen much progress that utilizing neural network approach to solve Cloze-style questions. In this paper, we present a novel model called attention-over-attention reader for the Cloze-style reading comprehension task. Our model aims to place another attention mechanism over the document-level attention, and induces "attended attention" for final predictions. Unlike the previous works, our neural network model requires less pre-defined hyper-parameters and uses an elegant architecture for modeling. Experimental results show that the proposed attention-over-attention model significantly outperforms various state-of-the-art systems by a large margin in public datasets, such as CNN and Children's Book Test datasets.

8+2 pages. accepted as a conference paper at ACL2017 (long paper)

Cited by in corpus (11)

• Attention in Natural Language Processing
• Syntactic Structure from Deep Learning
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Biomedical Question Answering: A Survey of Approaches and Challenges
• CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension
• Adversarial Explanations for Understanding Image Classification Decisions and Improved Neural Network Robustness
• Subsentence Extraction from Text Using Coverage-Based Deep Learning Language Models
• Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering
• Deep Understanding based Multi-Document Machine Reading Comprehension
• LECTOR: Summarizing E-book Reading Content for Personalized Student Support
• An Understanding-Oriented Robust Machine Reading Comprehension Model