Adversarial Generation of Natural Language

Abstract

Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.

arXiv comment11 pages, 3 figures, 5 tables

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Recurrent Neural Network Regularization
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Improved Training of Wasserstein GANs
• Language Modeling with Gated Convolutional Networks
• Sequence Level Training with Recurrent Neural Networks
• Adversarially Learned Inference
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Mode Regularized Generative Adversarial Networks
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Least Squares Generative Adversarial Networks
• Toward Controlled Generation of Text

Cited by in corpus (49)

• Recent Trends in Deep Learning Based Natural Language Processing
• Long Text Generation via Adversarial Training with Leaked Information
• An Introduction to Image Synthesis with Generative Adversarial Nets
• Clinically Accurate Chest X-Ray Report Generation
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Neural Text Generation: Past, Present and Beyond
• Adversarially Regularized Autoencoders
• CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
• Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and Perspectives
• A Tutorial on Deep Latent Variable Models of Natural Language
• Airline Passenger Name Record Generation using Generative Adversarial Networks
• Low-Resource Text Classification using Domain-Adversarial Learning
• Controllable Abstractive Summarization
• Evaluating Text GANs as Language Models
• Generative Adversarial Networks (GANs): What it can generate and What it cannot?
• SPI-GAN: Towards Single-Pixel Imaging through Generative Adversarial Network
• Synthetic Test Data Generation Using Recurrent Neural Networks: A Position Paper
• Categorical EHR Imputation with Generative Adversarial Nets
• ACtuAL: Actor-Critic Under Adversarial Learning
• Improved Natural Language Generation via Loss Truncation
• Recent Advances of Image Steganography with Generative Adversarial Networks
• Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Sometimes We Want Translationese
• Customizing an Adversarial Example Generator with Class-Conditional GANs
• Boosting Naturalness of Language in Task-oriented Dialogues via Adversarial Training
• Adversarial Neural Networks for Cross-lingual Sequence Tagging
• Automatic Repair and Type Binding of Undeclared Variables using Neural Networks
• Adversarial Sub-sequence for Text Generation
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• Practical License Plate Recognition in Unconstrained Surveillance Systems with Adversarial Super-Resolution
• Functorial Language Games for Question Answering
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• A Novel Framework for Selection of GANs for an Application
• Curriculum Learning for Deep Generative Models with Clustering
• Adversarial Learning of Semantic Relevance in Text to Image Synthesis
• ReGAN: RE[LAX|BAR|INFORCE] based Sequence Generation using GANs
• Distributional Discrepancy: A Metric for Unconditional Text Generation
• Curriculum Learning with Diversity for Supervised Computer Vision Tasks
• Language Modeling with Generative Adversarial Networks
• Topic-to-Essay Generation with Comprehensive Knowledge Enhancement
• Adversarial Machine Learning in Text Analysis and Generation
• The Detection of Distributional Discrepancy for Text Generation
• Generative Adversarial Nets for Multiple Text Corpora
• Evaluating Computational Language Models with Scaling Properties of Natural Language
• Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?
• TransSent: Towards Generation of Structured Sentences with Discourse Marker
• When Relation Networks meet GANs: Relation GANs with Triplet Loss
• Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis