On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
arXiv:1909.03186
Abstract
We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization. We perform a simple extractive step before generating a summary, which is then used to condition the transformer language model on relevant information before being tasked with generating a summary. We show that this extractive step significantly improves summarization results. We also show that this approach produces more abstractive summaries compared to prior work that employs a copy mechanism while still achieving higher rouge scores. Note: The abstract above was not written by the authors, it was generated by one of the models presented in this paper.
References in corpus (15)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- On the difficulty of training Recurrent Neural Networks
- WaveNet: A Generative Model for Raw Audio
- Convolutional Sequence to Sequence Learning
- Mixed Precision Training
- SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
- On Using Monolingual Corpora in Neural Machine Translation
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- Neural Machine Translation in Linear Time
- Incorporating Copying Mechanism in Sequence-to-Sequence Learning
- Neural Summarization by Extracting Sentences and Words
- Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting
- Classify or Select: Neural Architectures for Extractive Document Summarization
- Controlling Decoding for More Abstractive Summaries with Copy-Based Networks
Cited by in corpus (11)
- PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
- Big Bird: Transformers for Longer Sequences
- Recursively Summarizing Books with Human Feedback
- Neural Language Generation: Formulation, Methods, and Evaluation
- Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents
- SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization
- Dimsum @LaySumm 20: BART-based Approach for Scientific Document Summarization
- Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting
- Transform and Tell: Entity-Aware News Image Captioning
- Semantic Similarity Measure of Natural Language Text through Machine Learning and a Keyword-Aware Cross-Encoder-Ranking Summarizer -- A Case Study Using UCGIS GIS&T Body of Knowledge
- Learning to Summarize Long Texts with Memory Compression and Transfer