Learning to Decode for Future Success

Abstract

We introduce a simple, general strategy to manipulate the behavior of a neural decoder that enables it to generate outputs that have specific properties of interest (e.g., sequences of a pre-specified length). The model can be thought of as a simple version of the actor-critic model that uses an interpolation of the actor (the MLE-based token generation policy) and the critic (a value function that estimates the future values of the desired property) for decision making. We demonstrate that the approach is able to incorporate a variety of properties that cannot be handled by standard neural sequence decoders, such as sequence length and backward probability (probability of sources given targets), in addition to yielding consistent improvements in abstractive summarization and machine translation when the property to be optimized is BLEU or ROUGE scores.

References in corpus (14)

• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Microsoft COCO Captions: Data Collection and Evaluation Server
• A Neural Conversational Model
• Convolutional Neural Networks for Sentence Classification
• On Using Monolingual Corpora in Neural Machine Translation
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation
• An Actor-Critic Algorithm for Sequence Prediction
• Sequence-to-Sequence Learning as Beam-Search Optimization
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• LSTM based Conversation Models
• Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
• Minimum Risk Training for Neural Machine Translation
• Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models

Cited by in corpus (20)

• ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
• What makes a good conversation? How controllable attributes affect human judgments
• Trainable Greedy Decoding for Neural Machine Translation
• Controlling Decoding for More Abstractive Summaries with Copy-Based Networks
• Non-Autoregressive Neural Dialogue Generation
• Dynamic Past and Future for Neural Machine Translation
• Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
• Synchronous Bidirectional Inference for Neural Sequence Generation
• Teaching Machines to Converse
• Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity
• A Stable and Effective Learning Strategy for Trainable Greedy Decoding
• From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction
• Modeling Past and Future for Neural Machine Translation
• Cue-word Driven Neural Response Generation with a Shrinking Vocabulary
• Synchronous Bidirectional Neural Machine Translation
• Multi-Turn Beam Search for Neural Dialogue Modeling
• Global-aware Beam Search for Neural Abstractive Summarization
• Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
• Single-Queue Decoding for Neural Machine Translation
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses