Toward Diverse Text Generation with Inverse Reinforcement Learning

Abstract

Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, in this paper, we employ inverse reinforcement learning (IRL) for text generation. Specifically, the IRL framework learns a reward function on training data, and then an optimal policy to maximum the expected total reward. Similar to the adversarial models, the reward and policy function in IRL are optimized alternately. Our method has two advantages: (1) the reward function can produce more dense reward signals. (2) the generation policy, trained by "entropy regularized" policy gradient, encourages to generate more diversified texts. Experiment results demonstrate that our proposed method can generate higher quality texts than the previous methods.

arXiv comment7 pages, IJCAI 2018

References in corpus (8)

• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Improved Techniques for Training GANs
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• Unrolled Generative Adversarial Networks
• Bridging the Gap Between Value and Policy Based Reinforcement Learning
• GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
• Learning Stochastic Recurrent Networks
• Long Text Generation via Adversarial Training with Leaked Information

Cited by in corpus (10)

• A survey on text generation using generative adversarial networks
• Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges
• SALSA-TEXT : self attentive latent space based adversarial text generation
• Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model
• Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
• Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
• What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
• Conditional Text Generation for Harmonious Human-Machine Interaction
• Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation
• Molecular Attributes Transfer from Non-Parallel Data