A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

Abstract

We propose a unified model combining the strength of extractive and abstractive summarization. On the one hand, a simple extractive model can obtain sentence-level attention with high ROUGE scores but less readable. On the other hand, a more complicated abstractive model can obtain word-level dynamic attention to generate a more readable paragraph. In our model, sentence-level attention is used to modulate the word-level attention such that words in less attended sentences are less likely to be generated. Moreover, a novel inconsistency loss function is introduced to penalize the inconsistency between two levels of attentions. By end-to-end training our model with the inconsistency loss and original losses of extractive and abstractive models, we achieve state-of-the-art ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation.

9 pages, ACL 2018 oral. Project page: https://hsuwanting.github.io/unified_summ/. Code: https://github.com/HsuWanTing/unified-summarization

References in corpus (12)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Teaching Machines to Read and Comprehend
• A Deep Reinforced Model for Abstractive Summarization
• Sequence Level Training with Recurrent Neural Networks
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• Neural Summarization by Extracting Sentences and Words
• Classify or Select: Neural Architectures for Extractive Document Summarization
• Graph-based Neural Multi-Document Summarization
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Generative Adversarial Network for Abstractive Text Summarization
• Controllable Abstractive Summarization

Cited by in corpus (7)

• Concept Pointer Network for Abstractive Summarization
• DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
• The Style-Content Duality of Attractiveness: Learning to Write Eye-Catching Headlines via Disentanglement
• PoinT-5: Pointer Network and T-5 based Financial NarrativeSummarisation
• Salience Estimation with Multi-Attention Learning for Abstractive Text Summarization
• Selective Attention Encoders by Syntactic Graph Convolutional Networks for Document Summarization
• Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization