Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting
arXiv:1805.11080
Abstract
Inspired by how humans summarize long documents, we propose an accurate and fast summarization model that first selects salient sentences and then rewrites them abstractively (i.e., compresses and paraphrases) to generate a concise overall summary. We use a novel sentence-level policy gradient method to bridge the non-differentiable computation between these two neural networks in a hierarchical way, while maintaining language fluency. Empirically, we achieve the new state-of-the-art on all metrics (including human evaluation) on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores. Moreover, by first operating at the sentence-level and then the word-level, we enable parallel decoding of our neural generative model that results in substantially faster (10-20x) inference speed as well as 4x faster training convergence than previous long-paragraph encoder-decoder models. We also demonstrate the generalization of our model on the test-only DUC-2002 dataset, where we achieve higher scores than a state-of-the-art model.
arXiv commentACL 2018 (17 pages)
References in corpus (8)
- A Deep Reinforced Model for Abstractive Summarization
- SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
- Selective Encoding for Abstractive Sentence Summarization
- Incorporating Copying Mechanism in Sequence-to-Sequence Learning
- A Simple, Fast Diverse Decoding Algorithm for Neural Generation
- Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
- Language as a Latent Variable: Discrete Generative Models for Sentence Compression
- Abstractive Multi-Document Summarization via Phrase Selection and Merging
Cited by in corpus (31)
- Evaluation of Text Generation: A Survey
- Structured Neural Summarization
- Efficient Adaptation of Pretrained Transformers for Abstractive Summarization
- Length-controllable Abstractive Summarization by Guiding with Summary Prototype
- On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
- Summarization, Simplification, and Generation: The Case of Patents
- Robust Neural Abstractive Summarization Systems and Evaluation against Adversarial Information
- Improving Multi-turn Dialogue Modelling with Utterance ReWriter
- An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation
- SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization
- Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization
- AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
- In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
- Learning to Summarize Radiology Findings
- Concept Pointer Network for Abstractive Summarization
- Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View
- Controllable Abstractive Dialogue Summarization with Sketch Supervision
- DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
- Reinforced Extractive Summarization with Question-Focused Rewards
- PoBRL: Optimizing Multi-Document Summarization by Blending Reinforcement Learning Policies
- Subjective Bias in Abstractive Summarization
- Abstractive Text Summarization by Incorporating Reader Comments
- SportsSum2.0: Generating High-Quality Sports News from Live Text Commentary
- A more abstractive summarization model
- PoinT-5: Pointer Network and T-5 based Financial NarrativeSummarisation
- Augmented Abstractive Summarization With Document-LevelSemantic Graph
- Controllable Length Control Neural Encoder-Decoder via Reinforcement Learning
- Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
- Towards Controlled and Diverse Generation of Article Comments
- Latent Dirichlet Allocation with Residual Convolutional Neural Network Applied in Evaluating Credibility of Chinese Listed Companies
- Learning to Summarize Long Texts with Memory Compression and Transfer