Densely Connected Attention Propagation for Reading Comprehension

Abstract

We propose DecaProp (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). There are two distinct characteristics of our model. Firstly, our model densely connects all pairwise layers of the network, modeling relationships between passage and query across all hierarchical levels. Secondly, the dense connectors in our network are learned via attention instead of standard residual skip-connectors. To this end, we propose novel Bidirectional Attention Connectors (BAC) for efficiently forging connections throughout the network. We conduct extensive experiments on four challenging RC benchmarks. Our proposed approach achieves state-of-the-art results on all four, outperforming existing baselines by up to $2.6\%-14.2\%$ in absolute F1 score.

arXiv commentNIPS 2018

Cited by in corpus (14)

• Rethinking Search: Making Domain Experts out of Dilettantes
• Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
• Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents
• Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
• PQuAD: A Persian Question Answering Dataset
• Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
• A Discrete Hard EM Approach for Weakly Supervised Question Answering
• Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension
• Multi-style Generative Reading Comprehension
• Explicit Utilization of General Knowledge in Machine Reading Comprehension
• Controlling Risk of Web Question Answering
• Multi-span Style Extraction for Generative Reading Comprehension
• Relation Module for Non-answerable Prediction on Question Answering
• Context-Aware Answer Extraction in Question Answering