Neural Discourse Relation Recognition with Semantic Memory
arXiv:1603.03873 · doi:10.1016/j.neucom.2017.09.074
Abstract
Humans comprehend the meanings and relations of discourses heavily relying on their semantic memory that encodes general knowledge about concepts and facts. Inspired by this, we propose a neural recognizer for implicit discourse relation analysis, which builds upon a semantic memory that stores knowledge in a distributed fashion. We refer to this recognizer as SeMDER. Starting from word embeddings of discourse arguments, SeMDER employs a shallow encoder to generate a distributed surface representation for a discourse. A semantic encoder with attention to the semantic memory matrix is further established over surface representations. It is able to retrieve a deep semantic meaning representation for the discourse from the memory. Using the surface and semantic representations as input, SeMDER finally predicts implicit discourse relations via a neural recognizer. Experiments on the benchmark data set show that SeMDER benefits from the semantic memory and achieves substantial improvements of 2.56\% on average over current state-of-the-art baselines in terms of F1-score.
arXiv comment7 pages
References in corpus (8)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Proceedings of the 29th International Conference on Machine Learning (ICML-12)
- Recurrent Models of Visual Attention
- Effective Approaches to Attention-based Neural Machine Translation
- Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
- A Convolutional Neural Network for Modelling Sentences
- Variational Neural Discourse Relation Recognizer