Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models

Abstract

Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several multi-level structures to learn a VAE model to generate long, and coherent text. In particular, we use a hierarchy of stochastic layers between the encoder and decoder networks to generate more informative latent codes. We also investigate a multi-level decoder structure to learn a coherent long-term structure by generating intermediate sentence representations as high-level plan vectors. Empirical results demonstrate that a multi-level VAE model produces more coherent and less repetitive long text compared to the standard VAE models and can further mitigate the posterior-collapse issue.

To appear at ACL 2019

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• MaskGAN: Better Text Generation via Filling in the______
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
• Fast Decoding in Sequence Models using Discrete Latent Variables
• Texygen: A Benchmarking Platform for Text Generation Models
• DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
• A Retrieve-and-Edit Framework for Predicting Structured Outputs
• Latent Alignment and Variational Attention
• Adversarial Text Generation via Feature-Mover's Distance
• Topic-Guided Variational Autoencoders for Text Generation
• Generative Neural Machine Translation
• Deconvolutional Latent-Variable Model for Text Sequence Matching
• Eval all, trust a few, do wrong to none: Comparing sentence generation models
• Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation

Cited by in corpus (5)

• Controllable Paraphrase Generation with a Syntactic Exemplar
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• Deep Learning Models in Software Requirements Engineering
• Syntax-Infused Variational Autoencoder for Text Generation
• TransSent: Towards Generation of Structured Sentences with Discourse Marker