Adversarially Regularized Autoencoders

Abstract

Deep latent variable models, trained using variational autoencoders or generative adversarial networks, are now a key technique for representation learning of continuous structures. However, applying similar methods to discrete structures, such as text sequences or discretized images, has proven to be more challenging. In this work, we propose a flexible method for training deep latent variable models of discrete structures. Our approach is based on the recently-proposed Wasserstein autoencoder (WAE) which formalizes the adversarial autoencoder (AAE) as an optimal transport problem. We first extend this framework to model discrete sequences, and then further explore different learned priors targeting a controllable representation. This adversarially regularized autoencoder (ARAE) allows us to generate natural textual outputs as well as perform manipulations in the latent space to induce change in the output space. Finally we show that the latent representation can be trained to perform unaligned textual style transfer, giving improvements both in automatic/human evaluation compared to existing methods.

ICML 2018

References in corpus (27)

• Auto-Encoding Variational Bayes
• Spectral Normalization for Generative Adversarial Networks
• Categorical Reparameterization with Gumbel-Softmax
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Improved Training of Wasserstein GANs
• Adversarial Feature Learning
• Adversarially Learned Inference
• Style Transfer from Non-Parallel Text by Cross-Alignment
• A large annotated corpus for learning natural language inference
• Variational Lossy Autoencoder
• Unsupervised Learning of Disentangled Representations from Video
• Adversarial Learning for Neural Dialogue Generation
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Improving Variational Inference with Inverse Autoregressive Flow
• TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
• Boundary-Seeking Generative Adversarial Networks
• Adversarial Generation of Natural Language
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• On Unifying Deep Generative Models
• Language Generation with Recurrent Generative Adversarial Networks without Pre-training
• Topic Compositional Neural Language Model
• Unsupervised Text Style Transfer using Language Models as Discriminators
• A Hybrid Convolutional Variational Autoencoder for Text Generation
• Generating Sentences by Editing Prototypes
• Eval all, trust a few, do wrong to none: Comparing sentence generation models

Cited by in corpus (19)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
• Generating Multi-Categorical Samples with Generative Adversarial Networks
• Spherical Latent Spaces for Stable Variational Autoencoders
• Low-Resource Text Classification using Domain-Adversarial Learning
• Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation
• Improving Missing Data Imputation with Deep Generative Models
• CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Records
• SALSA-TEXT : self attentive latent space based adversarial text generation
• Correlated discrete data generation using adversarial training
• Out-of-domain Detection for Natural Language Understanding in Dialog Systems
• ACtuAL: Actor-Critic Under Adversarial Learning
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
• Fine-grained Sentiment Controlled Text Generation
• Conditional Hybrid GAN for Sequence Generation
• Generating Continuous Representations of Medical Texts
• Topic Modeling with Wasserstein Autoencoders
• Adversarial Decomposition of Text Representation
• Stacked Wasserstein Autoencoder