Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension
arXiv:1908.10721
Abstract
In this work, we propose to use linguistic annotations as a basis for a \textit{Discourse-Aware Semantic Self-Attention} encoder that we employ for reading comprehension on long narrative texts. We extract relations between discourse units, events and their arguments as well as coreferring mentions, using available annotation tools. Our empirical evaluation shows that the investigated structures improve the overall performance, especially intra-sentential and cross-sentential discourse relations, sentence-internal semantic role relations, and long-distance coreference relations. We show that dedicating self-attention heads to intra-sentential relations and relations connecting neighboring sentences is beneficial for finding answers to questions in longer contexts. Our findings encourage the use of discourse-semantic annotations to enhance the generalization capacity of self-attention models for reading comprehension.
arXiv commentAccepted as a long conference paper to EMNLP-IJCNLP 2019
References in corpus (8)
- Teaching Machines to Read and Comprehend
- The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
- One Model To Learn Them All
- Depthwise Separable Convolutions for Neural Machine Translation
- Dynamic Integration of Background Knowledge in Neural NLU Systems
- Question Answering as Global Reasoning over Semantic Abstractions
- Densely Connected Attention Propagation for Reading Comprehension
- Linguistic Knowledge as Memory for Recurrent Neural Networks
Cited by in corpus (5)
- LIMIT-BERT : Linguistic Informed Multi-Task BERT
- Multi-tasking Dialogue Comprehension with Discourse Parsing
- DAGN: Discourse-Aware Graph Network for Logical Reasoning
- Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading
- Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT