Adversarial Feature Matching for Text Generation

Abstract

The Generative Adversarial Network (GAN) has achieved great success in generating realistic (real-valued) synthetic data. However, convergence issues and difficulties dealing with discrete data hinder the applicability of GAN to text. We propose a framework for generating realistic text via adversarial training. We employ a long short-term memory network as generator, and a convolutional network as discriminator. Instead of using the standard objective of GAN, we propose matching the high-dimensional latent feature distributions of real and synthetic sentences, via a kernelized discrepancy metric. This eases adversarial training by alleviating the mode-collapsing problem. Our experiments show superior performance in quantitative evaluation, and demonstrate that our model can generate realistic-looking sentences.

Accepted by ICML 2017

References in corpus (20)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Improved Techniques for Training GANs
• Theano: new features and speed improvements
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Energy-based Generative Adversarial Network
• Adversarial Feature Learning
• Adversarially Learned Inference
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• A Convolutional Neural Network for Modelling Sentences
• Deep Reinforcement Learning for Dialogue Generation
• Unrolled Generative Adversarial Networks
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• Adversarial Learning for Neural Dialogue Generation
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
• Learning Generic Sentence Representations Using Convolutional Neural Networks
• On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives

Cited by in corpus (50)

• MaskGAN: Better Text Generation via Filling in the______
• Texygen: A Benchmarking Platform for Text Generation Models
• Language GANs Falling Short
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Neural Text Generation: Past, Present and Beyond
• Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems
• Creativity and Machine Learning: A Survey
• Defense Methods Against Adversarial Examples for Recurrent Neural Networks
• Generative Adversarial Network Training is a Continual Learning Problem
• Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
• JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets
• Adversarial Time-to-Event Modeling
• Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator
• Adversarial Text Generation Without Reinforcement Learning
• Semi-supervised Text Regression with Conditional Generative Adversarial Networks
• TorchGAN: A Flexible Framework for GAN Training and Evaluation
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• SALSA-TEXT : self attentive latent space based adversarial text generation
• Categorical EHR Imputation with Generative Adversarial Nets
• New Era of Deeplearning-Based Malware Intrusion Detection: The Malware Detection and Prediction Based On Deep Learning
• Comparing Generative Adversarial Network Techniques for Image Creation and Modification
• A Deep Generative Model for Reordering Adjacency Matrices
• Diverse Beam Search for Increased Novelty in Abstractive Summarization
• Generating Realistic Stock Market Order Streams
• Improving Movement Predictions of Traffic Actors in Bird's-Eye View Models using GANs and Differentiable Trajectory Rasterization
• Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings
• What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
• Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation
• Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text
• TreeGAN: Syntax-Aware Sequence Generation with Generative Adversarial Networks
• Say What? Collaborative Pop Lyric Generation Using Multitask Transfer Learning
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• DLGNet: A Transformer-based Model for Dialogue Response Generation
• Sequence Generation with Guider Network
• Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning
• Extractive Summary as Discrete Latent Variables
• Conditional Hybrid GAN for Sequence Generation
• Semi-supervised Conditional GAN for Simultaneous Generation and Detection of Phishing URLs: A Game theoretic Perspective
• Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Text Generation with Deep Variational GAN
• Towards Coherent and Cohesive Long-form Text Generation
• EgoCoder: Intelligent Program Synthesis with Hierarchical Sequential Neural Network Model
• Adversarial Bootstrapping for Dialogue Model Training
• Improving Text Generation with Student-Forcing Optimal Transport
• ChainGAN: A sequential approach to GANs
• ReGAN: RE[LAX|BAR|INFORCE] based Sequence Generation using GANs
• SparseGAN: Sparse Generative Adversarial Network for Text Generation
• Evaluating Computational Language Models with Scaling Properties of Natural Language
• Domain-adversarial Network Alignment