Actor-Critic based Training Framework for Abstractive Summarization

Abstract

We present a training framework for neural abstractive summarization based on actor-critic approaches from reinforcement learning. In the traditional neural network based methods, the objective is only to maximize the likelihood of the predicted summaries, no other assessment constraints are considered, which may generate low-quality summaries or even incorrect sentences. To alleviate this problem, we employ an actor-critic framework to enhance the training procedure. For the actor, we employ the typical attention based sequence-to-sequence (seq2seq) framework as the policy network for summary generation. For the critic, we combine the maximum likelihood estimator with a well designed global summary quality estimator which is a neural network based binary classifier aiming to make the generated summaries indistinguishable from the human-written ones. Policy gradient method is used to conduct the parameter learning. An alternating training strategy is proposed to conduct the joint training of the actor and critic models. Extensive experiments on some benchmark datasets in different languages show that our framework achieves improvements over the state-of-the-art methods.

arXiv comment10 pages. arXiv admin note: text overlap with arXiv:1708.00625

References in corpus (12)

• ADADELTA: An Adaptive Learning Rate Method
• Categorical Reparameterization with Gumbel-Softmax
• A Deep Reinforced Model for Abstractive Summarization
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• MaskGAN: Better Text Generation via Filling in the______
• Selective Encoding for Abstractive Sentence Summarization
• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• A Sentence Compression Based Framework to Query-Focused Multi-Document Summarization
• Adversarial Neural Machine Translation
• Distraction-Based Neural Networks for Document Summarization
• Controlling Output Length in Neural Encoder-Decoders
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression

Cited by in corpus (15)

• Abstractive Text Summarization: State of the Art, Challenges, and Improvements
• Neural Abstractive Text Summarization with Sequence-to-Sequence Models
• Abstractive Text Summarization Using the BRIO Training Paradigm
• Abstractive Summarization Using Attentive Neural Techniques
• Answers Unite! Unsupervised Metrics for Reinforced Summarization Models
• Sequence Generation with Guider Network
• PoBRL: Optimizing Multi-Document Summarization by Blending Reinforcement Learning Policies
• Multi-stage Pretraining for Abstractive Summarization
• PoinT-5: Pointer Network and T-5 based Financial NarrativeSummarisation
• Salience Estimation with Multi-Attention Learning for Abstractive Text Summarization
• Contrastive Attention Mechanism for Abstractive Sentence Summarization
• A Hybrid Word-Character Approach to Abstractive Summarization
• Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning
• Improving Adversarial Text Generation by Modeling the Distant Future
• GRET: Global Representation Enhanced Transformer