Topic-Guided Variational Autoencoders for Text Generation
arXiv:1903.07137
Abstract
We propose a topic-guided variational autoencoder (TGVAE) model for text generation. Distinct from existing variational autoencoder (VAE) based approaches, which assume a simple Gaussian prior for the latent code, our model specifies the prior as a Gaussian mixture model (GMM) parametrized by a neural topic module. Each mixture component corresponds to a latent topic, which provides guidance to generate sentences under the topic. The neural topic module and the VAE-based neural sequence module in our model are learned jointly. In particular, a sequence of invertible Householder transformations is applied to endow the approximate posterior of the latent code with high flexibility during model inference. Experimental results show that our TGVAE outperforms alternative approaches on both unconditional and conditional text generation, which can generate semantically-meaningful sentences with various topics.
References in corpus (11)
- Sequence to Sequence Learning with Neural Networks
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Semi-Supervised Learning with Deep Generative Models
- DRAW: A Recurrent Neural Network For Image Generation
- Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
- Variational Autoencoder for Deep Learning of Images, Labels and Captions
- Long Text Generation via Adversarial Training with Leaked Information
- A Neural Knowledge Language Model
- Improving Variational Auto-Encoders using Householder Flow
- A Hybrid Convolutional Variational Autoencoder for Text Generation
- Language as a Latent Variable: Discrete Generative Models for Sentence Compression