GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution

Abstract

Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements. The reason for this is that samples from a distribution on discrete objects such as the multinomial are not differentiable with respect to the distribution parameters. This problem can be avoided by using the Gumbel-softmax distribution, which is a continuous approximation to a multinomial distribution parameterized in terms of the softmax function. In this work, we evaluate the performance of GANs based on recurrent neural networks with Gumbel-softmax output distributions in the task of generating sequences of discrete elements.

References in corpus (1)

• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks

Cited by in corpus (35)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Grammar Variational Autoencoder
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Unsupervised Cipher Cracking Using Discrete GANs
• Neural Language Generation: Formulation, Methods, and Evaluation
• Improving Missing Data Imputation with Deep Generative Models
• Self-Adversarial Learning with Comparative Discrimination for Text Generation
• Categorical EHR Imputation with Generative Adversarial Nets
• Dynamics of Fourier Modes in Torus Generative Adversarial Networks
• Semantic Bottleneck Scene Generation
• Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
• Reward Constrained Interactive Recommendation with Natural Language Feedback
• Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning
• Conditional Hybrid GAN for Sequence Generation
• Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation
• Investigation of Sentiment Controllable Chatbot
• Automatically Generating Macro Research Reports from a Piece of News
• Enforcing Reasoning in Visual Commonsense Reasoning
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Russian Natural Language Generation: Creation of a Language Modelling Dataset and Evaluation with Modern Neural Architectures
• On Detecting Data Pollution Attacks On Recommender Systems Using Sequential GANs
• Wasserstein Learning of Determinantal Point Processes
• Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints
• Generative Models for Security: Attacks, Defenses, and Opportunities
• Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU
• Fast Generating A Large Number of Gumbel-Max Variables
• TaylorGAN: Neighbor-Augmented Policy Update for Sample-Efficient Natural Language Generation
• End-to-End Learning Using Cycle Consistency for Image-to-Caption Transformations
• Improving Adversarial Text Generation by Modeling the Distant Future
• OPAL-Net: A Generative Model for Part-based Object Layout Generation
• AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions
• Minority Class Oversampling for Tabular Data with Deep Generative Models
• AriEL: volume coding for sentence generation
• Adversarial Machine Learning in Text Analysis and Generation
• Towards Zero-Shot Knowledge Distillation for Natural Language Processing