Controlling Decoding for More Abstractive Summaries with Copy-Based Networks

Abstract

Attention-based neural abstractive summarization systems equipped with copy mechanisms have shown promising results. Despite this success, it has been noticed that such a system generates a summary by mostly, if not entirely, copying over phrases, sentences, and sometimes multiple consecutive sentences from an input paragraph, effectively performing extractive summarization. In this paper, we verify this behavior using the latest neural abstractive summarization system - a pointer-generator network. We propose a simple baseline method that allows us to control the amount of copying without retraining. Experiments indicate that the method provides a strong baseline for abstractive systems looking to obtain high ROUGE scores while minimizing overlap with the source article, substantially reducing the n-gram overlap with the original article while keeping within 2 points of the original model's ROUGE score.

References in corpus (8)

• Teaching Machines to Read and Comprehend
• A Deep Reinforced Model for Abstractive Summarization
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation
• Get To The Point: Summarization with Pointer-Generator Networks
• Challenges in Data-to-Document Generation
• Learning to Decode for Future Success
• Search Engine Guided Non-Parametric Neural Machine Translation
• Trainable Greedy Decoding for Neural Machine Translation

Cited by in corpus (3)

• On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
• Point-less: More Abstractive Summarization with Pointer-Generator Networks
• Concept Pointer Network for Abstractive Summarization