In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes
arXiv:1909.10852 · doi:10.18653/v1/K19-1077
Abstract
Various Seq2Seq learning models designed for machine translation were applied for abstractive summarization task recently. Despite these models provide high ROUGE scores, they are limited to generate comprehensive summaries with a high level of abstraction due to its degenerated attention distribution. We introduce Diverse Convolutional Seq2Seq Model(DivCNN Seq2Seq) using Determinantal Point Processes methods(Micro DPPs and Macro DPPs) to produce attention distribution considering both quality and diversity. Without breaking the end to end architecture, DivCNN Seq2Seq achieves a higher level of comprehensiveness compared to vanilla models and strong baselines. All the reproducible codes and datasets are available online.
arXiv comment11 pages, 7 figures, 4 tables
References in corpus (17)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Teaching Machines to Read and Comprehend
- Convolutional Sequence to Sequence Learning
- Language Modeling with Gated Convolutional Networks
- Recurrent Models of Visual Attention
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- WikiHow: A Large Scale Text Summarization Dataset
- Get To The Point: Summarization with Pointer-Generator Networks
- Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting
- Fast Greedy MAP Inference for Determinantal Point Process to Improve Recommendation Diversity
- Distraction-Based Neural Networks for Document Summarization
- Abstractive Summarization of Reddit Posts with Multi-level Memory Networks
- Bottom-Up Abstractive Summarization
- BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization