A Hybrid Convolutional Variational Autoencoder for Text Generation

Abstract

In this paper we explore the effect of architectural choices on learning a Variational Autoencoder (VAE) for text generation. In contrast to the previously introduced VAE model for text where both the encoder and decoder are RNNs, we propose a novel hybrid architecture that blends fully feed-forward convolutional and deconvolutional components with a recurrent language model. Our architecture exhibits several attractive properties such as faster run time and convergence, ability to better handle long sequences and, more importantly, it helps to avoid some of the major difficulties posed by training VAE models on textual data.

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Neural Architecture Search with Reinforcement Learning
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Language Modeling with Gated Convolutional Networks
• Exploring the Limits of Language Modeling
• Autoencoding beyond pixels using a learned similarity metric
• Learning Deconvolution Network for Semantic Segmentation
• Layer Normalization
• Neural Machine Translation in Linear Time
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Neural Variational Inference for Text Processing
• Show and Tell: A Neural Image Caption Generator
• Attribute2Image: Conditional Image Generation from Visual Attributes
• Techniques for Learning Binary Stochastic Feedforward Neural Networks

Cited by in corpus (32)

• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• Toward Controlled Generation of Text
• BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling
• From Variational to Deterministic Autoencoders
• On Accurate Evaluation of GANs for Language Generation
• Deconvolutional Paragraph Representation Learning
• Topic-Guided Variational Autoencoders for Text Generation
• Deblending galaxies with Variational Autoencoders: a joint multi-band, multi-instrument approach
• Image Captioning at Will: A Versatile Scheme for Effectively Injecting Sentiments into Image Descriptions
• An advanced hybrid deep adversarial autoencoder for parameterized nonlinear fluid flow modelling
• Adversarially Regularized Autoencoders
• Semi-Amortized Variational Autoencoders
• Eval all, trust a few, do wrong to none: Comparing sentence generation models
• Z-Forcing: Training Stochastic Recurrent Networks
• A Tutorial on Deep Latent Variable Models of Natural Language
• Improving Variational Encoder-Decoders in Dialogue Generation
• Spherical Latent Spaces for Stable Variational Autoencoders
• A Conditional Variational Framework for Dialog Generation
• Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
• Dirichlet Variational Autoencoder for Text Modeling
• Abstractive Text Classification Using Sequence-to-convolution Neural Networks
• Exploiting Deep Learning for Persian Sentiment Analysis
• A Batch Normalized Inference Network Keeps the KL Vanishing Away
• UVA: A Universal Variational Framework for Continuous Age Analysis
• Controllable and Diverse Text Generation in E-commerce
• Variational Auto-Encoder Based Approximate Bayesian Computation Uncertian Inverse Method for Sheet Metal Forming Problem
• Improving latent variable descriptiveness with AutoGen
• Learning a Representation Map for Robot Navigation using Deep Variational Autoencoder
• Language Transfer of Audio Word2Vec: Learning Audio Segment Representations without Target Language Data
• Bayesian neural networks and dimensionality reduction
• Dual Latent Variable Model for Low-Resource Natural Language Generation in Dialogue Systems
• Deep Generative Models: Deterministic Prediction with an Application in Inverse Rendering