Get To The Point: Summarization with Pointer-Generator Networks

Abstract

Neural sequence-to-sequence models have provided a viable new approach for abstractive text summarization (meaning they are not restricted to simply selecting and rearranging passages from the original text). However, these models have two shortcomings: they are liable to reproduce factual details inaccurately, and they tend to repeat themselves. In this work we propose a novel architecture that augments the standard sequence-to-sequence attentional model in two orthogonal ways. First, we use a hybrid pointer-generator network that can copy words from the source text via pointing, which aids accurate reproduction of information, while retaining the ability to produce novel words through the generator. Second, we use coverage to keep track of what has been summarized, which discourages repetition. We apply our model to the CNN / Daily Mail summarization task, outperforming the current abstractive state-of-the-art by at least 2 ROUGE points.

Add METEOR evaluation results, add some citations, fix some equations (what are now equations 1, 8 and 11 were missing a bias term), fix url to pyrouge package, add acknowledgments

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
• Pointer Sentinel Mixture Models
• Efficient Summarization with Read-Again and Copy Mechanism
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

Cited by in corpus (87)

• Fine-Tuning Language Models from Human Preferences
• Big Bird: Transformers for Longer Sequences
• MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization
• ESPnet: End-to-End Speech Processing Toolkit
• Topic-Guided Variational Autoencoders for Text Generation
• The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection
• UHop: An Unrestricted-Hop Relation Extraction Framework for Knowledge-Based Question Answering
• Generating Bug-Fixes Using Pretrained Transformers
• Text Generation from Knowledge Graphs with Graph Transformers
• Code-Switching for Enhancing NMT with Pre-Specified Translation
• Learning to summarize from human feedback
• Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems
• Neural Text Generation: A Practical Guide
• Monotonic Chunkwise Attention
• Addressing Some Limitations of Transformers with Feedback Memory
• Learning to Extract Coherent Summary via Deep Reinforcement Learning
• Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data
• Query-Based Abstractive Summarization Using Neural Networks
• Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking
• GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension
• The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Results
• Relevance Transformer: Generating Concise Code Snippets with Relevance Feedback
• Learning by Semantic Similarity Makes Abstractive Summarization Better
• Exploration on Generating Traditional Chinese Medicine Prescription from Symptoms with an End-to-End method
• Improving Multi-turn Dialogue Modelling with Utterance ReWriter
• Topic Augmented Generator for Abstractive Summarization
• Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA
• Strategies for Structuring Story Generation
• Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
• Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
• Natural Question Generation with Reinforcement Learning Based Graph-to-Sequence Model
• An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generation
• Controlling Decoding for More Abstractive Summaries with Copy-Based Networks
• Recursive Graphical Neural Networks for Text Classification
• In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
• AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
• Entity Commonsense Representation for Neural Abstractive Summarization
• Discriminative Adversarial Search for Abstractive Summarization
• The SPPD System for Schema Guided Dialogue State Tracking Challenge
• Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
• Understanding Multi-Head Attention in Abstractive Summarization
• Dimsum @LaySumm 20: BART-based Approach for Scientific Document Summarization
• Improved Natural Language Generation via Loss Truncation
• MAST: A Memory-Augmented Self-supervised Tracker
• Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
• Point-less: More Abstractive Summarization with Pointer-Generator Networks
• Automatic Generation of Chinese Short Product Titles for Mobile Display
• Controllable Abstractive Dialogue Summarization with Sketch Supervision
• Unsupervised Paraphrasing by Simulated Annealing
• Question Generation by Transformers
• Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View
• A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization
• Robust Dialogue Utterance Rewriting as Sequence Tagging
• Contextualizing ASR Lattice Rescoring with Hybrid Pointer Network Language Model
• AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization
• HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
• Denoising based Sequence-to-Sequence Pre-training for Text Generation
• Teaching Machines to Converse
• MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
• Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
• Combining Word Embeddings and N-grams for Unsupervised Document Summarization
• The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement
• Query-Focused EHR Summarization to Aid Imaging Diagnosis
• Easy-to-Hard: Leveraging Simple Questions for Complex Question Generation
• Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization
• Data Augmentation for Copy-Mechanism in Dialogue State Tracking
• Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
• A more abstractive summarization model
• Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization
• SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline
• A Study of the Tasks and Models in Machine Reading Comprehension
• Asking Complex Questions with Multi-hop Answer-focused Reasoning
• Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task
• Automatic Text Extractive Summarization Based on Graph and Pre-trained Language Model Attention
• Towards Controlled and Diverse Generation of Article Comments
• Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization
• Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations
• Multi-Head Decoder for End-to-End Speech Recognition
• Centrality Meets Centroid: A Graph-based Approach for Unsupervised Document Summarization
• Autoregressive Knowledge Distillation through Imitation Learning
• Conversational Semantic Role Labeling
• Point or Generate Dialogue State Tracker
• Open4Business(O4B): An Open Access Dataset for Summarizing Business Documents
• Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance
• Topic Modeling and Progression of American Digital News Media During the Onset of the COVID-19 Pandemic
• Cross Copy Network for Dialogue Generation
• Abstractive Summarization Improved by WordNet-based Extractive Sentences