A Latent Variable Recurrent Neural Network for Discourse Relation Language Models

Abstract

This paper presents a novel latent variable recurrent neural network architecture for jointly modeling sequences of words and (possibly latent) discourse relations between adjacent sentences. A recurrent neural network generates individual words, thus reaping the benefits of discriminatively-trained vector representations. The discourse relations are represented with a latent variable, which can be predicted or marginalized, depending on the task. The resulting model can therefore employ a training objective that includes not only discourse relation classification, but also word prediction. As a result, it outperforms state-of-the-art alternatives for two tasks: implicit discourse relation classification in the Penn Discourse Treebank, and dialog act classification in the Switchboard corpus. Furthermore, by marginalizing over latent discourse relations at test time, we obtain a discourse informed language model, which improves over a strong LSTM baseline.

NAACL 2016 camera ready, 11 pages

Cited by in corpus (16)

• Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
• Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks
• Local Contextual Attention with Hierarchical Structure for Dialogue Act Recognition
• Dialogue Act Recognition via CRF-Attentive Structured Network
• Neural Network Models for Implicit Discourse Relation Classification in English and Chinese without Surface Features
• Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion
• ISO-Standard Domain-Independent Dialogue Act Tagging for Conversational Agents
• Hierarchical Multi-Label Dialog Act Recognition on Spanish Data
• Acquiring Annotated Data with Cross-lingual Explicitation for Implicit Discourse Relation Classification
• Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph
• Discourse Relation Embeddings: Representing the Relations between Discourse Segments in Social Media
• Learning to Explicitate Connectives with Seq2Seq Network for Implicit Discourse Relation Classification
• Exploring Textual and Speech information in Dialogue Act Classification with Speaker Domain Adaptation
• What Changed Your Mind: The Roles of Dynamic Topics and Discourse in Argumentation Process
• Building A User-Centric and Content-Driven Socialbot
• Content Word-based Sentence Decoding and Evaluating for Open-domain Neural Response Generation