SALSA-TEXT : self attentive latent space based adversarial text generation
arXiv:1809.11155 · doi:10.1007/978-3-030-18305-9_10
Abstract
Inspired by the success of self attention mechanism and Transformer architecture in sequence transduction and image generation applications, we propose novel self attention-based architectures to improve the performance of adversarial latent code- based schemes in text generation. Adversarial latent code-based text generation has recently gained a lot of attention due to their promising results. In this paper, we take a step to fortify the architectures used in these setups, specifically AAE and ARAE. We benchmark two latent code-based methods (AAE and ARAE) designed based on adversarial setups. In our experiments, the Google sentence compression dataset is utilized to compare our method with these methods using various objective and subjective measures. The experiments demonstrate the proposed (self) attention-based models outperform the state-of-the-art in adversarial code-based text generation.
arXiv comment10 pages, 3 figures, under review at ICLR 2019
References in corpus (18)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- Spectral Normalization for Generative Adversarial Networks
- Categorical Reparameterization with Gumbel-Softmax
- Self-Attention Generative Adversarial Networks
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- Improved Training of Wasserstein GANs
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Universal Transformers
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
- Toward Controlled Generation of Text
- Adversarial Feature Matching for Text Generation
- Generating Text with Deep Reinforcement Learning
- Adversarially Regularized Autoencoders
- Eval all, trust a few, do wrong to none: Comparing sentence generation models
- Toward Diverse Text Generation with Inverse Reinforcement Learning
- Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method