Topic-Guided Variational Autoencoders for Text Generation

Abstract

We propose a topic-guided variational autoencoder (TGVAE) model for text generation. Distinct from existing variational autoencoder (VAE) based approaches, which assume a simple Gaussian prior for the latent code, our model specifies the prior as a Gaussian mixture model (GMM) parametrized by a neural topic module. Each mixture component corresponds to a latent topic, which provides guidance to generate sentences under the topic. The neural topic module and the VAE-based neural sequence module in our model are learned jointly. In particular, a sequence of invertible Householder transformations is applied to endow the approximate posterior of the latent code with high flexibility during model inference. Experimental results show that our TGVAE outperforms alternative approaches on both unconditional and conditional text generation, which can generate semantically-meaningful sentences with various topics.

References in corpus (29)

• Sequence to Sequence Learning with Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Convolutional Sequence to Sequence Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Semi-Supervised Learning with Deep Generative Models
• DRAW: A Recurrent Neural Network For Image Generation
• Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• Selective Encoding for Abstractive Sentence Summarization
• Discovering Discrete Latent Topics with Neural Variational Inference
• Adversarial Learning for Neural Dialogue Generation
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
• Long Text Generation via Adversarial Training with Leaked Information
• Texygen: A Benchmarking Platform for Text Generation Models
• A Neural Knowledge Language Model
• Get To The Point: Summarization with Pointer-Generator Networks
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering
• Improving Variational Auto-Encoders using Householder Flow
• Adversarial Text Generation via Feature-Mover's Distance
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Semi-Amortized Variational Autoencoders
• Variational Neural Machine Translation
• Distilled Wasserstein Learning for Word Embedding and Topic Modeling
• A Conditional Variational Framework for Dialog Generation
• A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization
• Sequence Generation with Guider Network

Cited by in corpus (22)

• Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery
• ControlVAE: Controllable Variational Autoencoder
• Deblending galaxies with Variational Autoencoders: a joint multi-band, multi-instrument approach
• DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
• Topic Augmented Generator for Abstractive Summarization
• Fast Cross-domain Data Augmentation through Neural Sentence Editing
• Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting
• Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
• Conditional Text Generation for Harmonious Human-Machine Interaction
• Reward Constrained Interactive Recommendation with Natural Language Feedback
• RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators
• CCGG: A Deep Autoregressive Model for Class-Conditional Graph Generation
• APo-VAE: Text Generation in Hyperbolic Space
• Syntax-Infused Variational Autoencoder for Text Generation
• Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder
• Controllable and Diverse Text Generation in E-commerce
• Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation
• Constrained Text Generation with Global Guidance -- Case Study on CommonGen
• ControlVAE: Tuning, Analytical Properties, and Performance Analysis
• Multiple Imputation for Biomedical Data using Monte Carlo Dropout Autoencoders
• EXoN: EXplainable encoder Network
• Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior