GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution

Abstract

Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements. The reason for this is that samples from a distribution on discrete objects such as the multinomial are not differentiable with respect to the distribution parameters. This problem can be avoided by using the Gumbel-softmax distribution, which is a continuous approximation to a multinomial distribution parameterized in terms of the softmax function. In this work, we evaluate the performance of GANs based on recurrent neural networks with Gumbel-softmax output distributions in the task of generating sequences of discrete elements.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Categorical Reparameterization with Gumbel-Softmax
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Generating Images with Perceptual Similarity Metrics based on Deep Networks

Cited by in corpus (76)

• Style Transfer from Non-Parallel Text by Cross-Alignment
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Synthesizing Tabular Data using Generative Adversarial Networks
• NetGAN: Generating Graphs via Random Walks
• Adversarial Ranking for Language Generation
• Texygen: A Benchmarking Platform for Text Generation Models
• Toward Controlled Generation of Text
• A survey on text generation using generative adversarial networks
• DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
• Grammar Variational Autoencoder
• Generative Models for Automatic Chemical Design
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• D-VAE: A Variational Autoencoder for Directed Acyclic Graphs
• Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
• Neural Text Generation: Past, Present and Beyond
• Generating Multi-Categorical Samples with Generative Adversarial Networks
• Towards Binary-Valued Gates for Robust LSTM Training
• Defense Methods Against Adversarial Examples for Recurrent Neural Networks
• Adversarially Regularized Autoencoders
• A Distributional Approach to Controlled Text Generation
• Toward Diverse Text Generation with Inverse Reinforcement Learning
• Unsupervised Cipher Cracking Using Discrete GANs
• Neural Language Generation: Formulation, Methods, and Evaluation
• Semi-supervised Text Regression with Conditional Generative Adversarial Networks
• PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences
• Improving Missing Data Imputation with Deep Generative Models
• A Review of Learning with Deep Generative Models from Perspective of Graphical Modeling
• Categorical EHR Imputation with Generative Adversarial Nets
• New Era of Deeplearning-Based Malware Intrusion Detection: The Malware Detection and Prediction Based On Deep Learning
• Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation
• Self-Adversarial Learning with Comparative Discrimination for Text Generation
• Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
• Dynamics of Fourier Modes in Torus Generative Adversarial Networks
• Semantic Bottleneck Scene Generation
• Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation
• Image Captioning Based on a Hierarchical Attention Mechanism and Policy Gradient Optimization
• Leveraging Long and Short-term Information in Content-aware Movie Recommendation
• Reward Constrained Interactive Recommendation with Natural Language Feedback
• Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning
• Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning
• Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
• Conditional Hybrid GAN for Sequence Generation
• Improving Reinforcement Learning Based Image Captioning with Natural Language Prior
• Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation
• Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation
• Invertible Gaussian Reparameterization: Revisiting the Gumbel-Softmax
• An Auxiliary Classifier Generative Adversarial Framework for Relation Extraction
• Investigation of Sentiment Controllable Chatbot
• Russian Natural Language Generation: Creation of a Language Modelling Dataset and Evaluation with Modern Neural Architectures
• Deep Learning on Attributed Graphs: A Journey from Graphs to Their Embeddings and Back
• Generative Models for Security: Attacks, Defenses, and Opportunities
• Enforcing Reasoning in Visual Commonsense Reasoning
• Automatically Generating Macro Research Reports from a Piece of News
• ReGAN: RE[LAX|BAR|INFORCE] based Sequence Generation using GANs
• Wasserstein Learning of Determinantal Point Processes
• On Detecting Data Pollution Attacks On Recommender Systems Using Sequential GANs
• Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints
• Bi-Directional Differentiable Input Reconstruction for Low-Resource Neural Machine Translation
• TaylorGAN: Neighbor-Augmented Policy Update for Sample-Efficient Natural Language Generation
• Improvement in Machine Translation with Generative Adversarial Networks
• Generating Symbolic Reasoning Problems with Transformer GANs
• Generating Multi-type Temporal Sequences to Mitigate Class-imbalanced Problem
• Towards Zero-Shot Knowledge Distillation for Natural Language Processing
• Differentiable Combinatorial Losses through Generalized Gradients of Linear Programs
• Natural Adversarial Sentence Generation with Gradient-based Perturbation
• OPAL-Net: A Generative Model for Part-based Object Layout Generation
• Adversarial Machine Learning in Text Analysis and Generation
• Seq2Seq Mimic Games: A Signaling Perspective
• Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU
• AriEL: volume coding for sentence generation
• Minority Class Oversampling for Tabular Data with Deep Generative Models
• Fast Generating A Large Number of Gumbel-Max Variables
• Improving Adversarial Text Generation by Modeling the Distant Future
• End-to-End Learning Using Cycle Consistency for Image-to-Caption Transformations
• AI-Powered Text Generation for Harmonious Human-Machine Interaction: Current State and Future Directions