Bottom-Up Abstractive Summarization

Abstract

Neural network-based methods for abstractive summarization produce outputs that are more fluent than other techniques, but which can be poor at content selection. This work proposes a simple technique for addressing this issue: use a data-efficient content selector to over-determine phrases in a source document that should be part of the summary. We use this selector as a bottom-up attention step to constrain the model to likely phrases. We show that this approach improves the ability to compress text, while still generating fluent summaries. This two-step process is both simpler and higher performing than other end-to-end content selection models, leading to significant improvements on ROUGE for both the CNN-DM and NYT corpus. Furthermore, the content selector can be trained with as little as 1,000 sentences, making it easy to transfer a trained summarizer to a new domain.

arXiv commentEMNLP 2018

Cited by in corpus (20)

• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• Pre-trained Language Model Representations for Language Generation
• Length-controllable Abstractive Summarization by Guiding with Summary Prototype
• PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
• QBSUM: a Large-Scale Query-Based Document Summarization Dataset from Real-world Applications
• Liputan6: A Large-scale Indonesian Dataset for Text Summarization
• Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization
• Multi-style Generative Reading Comprehension
• In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
• Understanding Multi-Head Attention in Abstractive Summarization
• Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries
• Unsupervised Pre-training for Natural Language Generation: A Literature Review
• Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
• Improved Document Modelling with a Neural Discourse Parser
• Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization
• Attend to the beginning: A study on using bidirectional attention for extractive summarization
• User-Inspired Posterior Network for Recommendation Reason Generation
• Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance
• Learning to Summarize Long Texts with Memory Compression and Transfer