Sequence Generation with Guider Network
arXiv:1811.00696
Abstract
Sequence generation with reinforcement learning (RL) has received significant attention recently. However, a challenge with such methods is the sparse-reward problem in the RL training process, in which a scalar guiding signal is often only available after an entire sequence has been generated. This type of sparse reward tends to ignore the global structural information of a sequence, causing generation of sequences that are semantically inconsistent. In this paper, we present a model-based RL approach to overcome this issue. Specifically, we propose a novel guider network to model the sequence-generation environment, which can assist next-word prediction and provide intermediate rewards for generator optimization. Extensive experiments show that the proposed method leads to improved performance for both unconditional and conditional sequence-generation tasks.
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Sequence to Sequence Learning with Neural Networks
- A Deep Reinforced Model for Abstractive Summarization
- Style Transfer from Non-Parallel Text by Cross-Alignment
- Model-Ensemble Trust-Region Policy Optimization
- Image Captioning with Semantic Attention
- An Actor-Critic Algorithm for Sequence Prediction
- Long Text Generation via Adversarial Training with Leaked Information
- Adversarial Feature Matching for Text Generation
- Twin Networks: Matching the Future for Sequence Generation
- Actor-Critic based Training Framework for Abstractive Summarization