DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
arXiv:1811.02394
Abstract
We propose DeepChannel, a robust, data-efficient, and interpretable neural model for extractive document summarization. Given any document-summary pair, we estimate a salience score, which is modeled using an attention-based deep neural network, to represent the salience degree of the summary for yielding the document. We devise a contrastive training strategy to learn the salience estimation network, and then use the learned salience score as a guide and iteratively extract the most salient sentences from the document as our generated summary. In experiments, our model not only achieves state-of-the-art ROUGE scores on CNN/Daily Mail dataset, but also shows strong robustness in the out-of-domain test on DUC2007 test set. Moreover, our model reaches a ROUGE-1 F-1 score of 39.41 on CNN/Daily Mail test set with merely training set, demonstrating a tremendous data efficiency.
arXiv commentAAAI-19
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Teaching Machines to Read and Comprehend
- A Deep Reinforced Model for Abstractive Summarization
- SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
- An efficient framework for learning sentence representations
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting
- Abstractive Multi-Document Summarization via Phrase Selection and Merging
- Learning to Extract Coherent Summary via Deep Reinforcement Learning
- Controllable Abstractive Summarization
- A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss