Dirichlet Variational Autoencoder for Text Modeling
arXiv:1811.00135
Abstract
We introduce an improved variational autoencoder (VAE) for text modeling with topic information explicitly modeled as a Dirichlet latent variable. By providing the proposed model topic awareness, it is more superior at reconstructing input texts. Furthermore, due to the inherent interactions between the newly introduced Dirichlet variable and the conventional multivariate Gaussian variable, the model is less prone to KL divergence vanishing. We derive the variational lower bound for the new model and conduct experiments on four different data sets. The results show that the proposed model is superior at text reconstruction across the latent space and classifications on learned representations have higher test accuracies.
References in corpus (15)
- Categorical Reparameterization with Gumbel-Softmax
- Convolutional Neural Networks for Sentence Classification
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- DRAW: A Recurrent Neural Network For Image Generation
- Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
- Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
- Toward Controlled Generation of Text
- Implicit Reparameterization Gradients
- Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
- Pathwise Derivatives Beyond the Reparameterization Trick
- A Hybrid Convolutional Variational Autoencoder for Text Generation
- Variational Neural Machine Translation
- The Generalized Reparameterization Gradient
- Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation
- Variational Knowledge Graph Reasoning
Cited by in corpus (7)
- Latent Space Factorisation and Manipulation via Matrix Subspace Projection
- Dirichlet Graph Variational Autoencoder
- Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
- Text Modeling with Syntax-Aware Variational Autoencoders
- Neural Gaussian Copula for Variational Autoencoder
- Discrete Auto-regressive Variational Attention Models for Text Modeling
- On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond