Adversarial Sub-sequence for Text Generation

Abstract

Generative adversarial nets (GAN) has been successfully introduced for generating text to alleviate the exposure bias. However, discriminators in these models only evaluate the entire sequence, which causes feedback sparsity and mode collapse. To tackle these problems, we propose a novel mechanism. It first segments the entire sequence into several sub-sequences. Then these sub-sequences, together with the entire sequence, are evaluated individually by the discriminator. At last these feedback signals are all used to guide the learning of GAN. This mechanism learns the generation of both the entire sequence and the sub-sequences simultaneously. Learning to generate sub-sequences is easy and is helpful in generating an entire sequence. It is easy to improve the existing GAN-based models with this mechanism. We rebuild three previous well-designed models with our mechanism, and the experimental results on benchmark data show these models are improved significantly, the best one outperforms the state-of-the-art model.\footnote[1]{All code and data are available at https://github.com/liyzcj/seggan.git

References in corpus (13)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Categorical Reparameterization with Gumbel-Softmax
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• MaskGAN: Better Text Generation via Filling in the______
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Mode Regularized Generative Adversarial Networks
• Adversarial Learning for Neural Dialogue Generation
• Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
• Long Text Generation via Adversarial Training with Leaked Information
• Texygen: A Benchmarking Platform for Text Generation Models
• Adversarial Generation of Natural Language
• On Accurate Evaluation of GANs for Language Generation