Towards Coherent and Cohesive Long-form Text Generation
arXiv:1811.00511
Abstract
Generating coherent and cohesive long-form texts is a challenging task. Previous works relied on large amounts of human-generated texts to train neural language models. However, few attempted to explicitly improve neural language models from the perspectives of coherence and cohesion. In this work, we propose a new neural language model that is equipped with two neural discriminators which provide feedback signals at the levels of sentence (cohesion) and paragraph (coherence). Our model is trained using a simple yet efficient variant of policy gradient, called negative-critical sequence training, which is proposed to eliminate the need of training a separate critic for estimating baseline. Results demonstrate the effectiveness of our approach, showing improvements over the strong baseline -- recurrent attention-based bidirectional MLE-trained neural language model.
arXiv commentSelected for spotlight oral presentation at NAACL-HLT 2019 Workshop on Narrative Understanding
References in corpus (11)
- Sequence to Sequence Learning with Neural Networks
- Natural Language Processing (almost) from Scratch
- A Neural Conversational Model
- MaskGAN: Better Text Generation via Filling in the______
- Deep Convolutional Ranking for Multilabel Image Annotation
- An Actor-Critic Algorithm for Sequence Prediction
- Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
- Long Text Generation via Adversarial Training with Leaked Information
- Adversarial Ranking for Language Generation
- Adversarial Feature Matching for Text Generation
- DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder