Neural Net Models for Open-Domain Discourse Coherence

Abstract

Discourse coherence is strongly associated with text quality, making it important to natural language generation and understanding. Yet existing models of coherence focus on measuring individual aspects of coherence (lexical overlap, rhetorical structure, entity centering) in narrow domains. In this paper, we describe domain-independent neural models of discourse coherence that are capable of measuring multiple aspects of coherence in existing sentences and can maintain coherence while generating new sentences. We study both discriminative models that learn to distinguish coherent from incoherent discourse, and generative models that produce coherent text, including a novel neural latent-variable Markovian generative model that captures the latent discourse dependencies between sentences in a text. Our work achieves state-of-the-art performance on multiple coherence evaluations, and marks an initial step in generating coherent texts given discourse contexts.

References in corpus (7)

• Sequence to Sequence Learning with Neural Networks
• ADADELTA: An Adaptive Learning Rate Method
• Tutorial on Variational Autoencoders
• A Hierarchical Neural Autoencoder for Paragraphs and Documents
• Learning Stochastic Recurrent Networks
• Adversarial Evaluation of Dialogue Models
• Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models

Cited by in corpus (9)

• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning
• End-to-End Neural Sentence Ordering Using Pointer Network
• Local and Global Context-Based Pairwise Models for Sentence Ordering
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation
• Joint Syntacto-Discourse Parsing and the Syntacto-Discourse Treebank
• A New Sentence Ordering Method Using BERT Pretrained Model
• Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation