MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization

Abstract

Abstractive summarization has been studied using neural sequence transduction methods with datasets of large, paired document-summary examples. However, such datasets are rare and the models trained from them do not generalize to other domains. Recently, some progress has been made in learning sequence-to-sequence mappings with only unpaired examples. In our work, we consider the setting where there are only documents (product or business reviews) with no summaries provided, and propose an end-to-end, neural model architecture to perform unsupervised abstractive summarization. Our proposed model consists of an auto-encoder where the mean of the representations of the input reviews decodes to a reasonable summary-review while not relying on any review-specific features. We consider variants of the proposed architecture and perform an ablation study to show the importance of specific components. We show through automated metrics and human evaluation that the generated summaries are highly abstractive, fluent, relevant, and representative of the average sentiment of the input reviews. Finally, we collect a reference evaluation dataset and show that our model outperforms a strong extractive baseline.

arXiv commentAccepted to ICML 2019

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Categorical Reparameterization with Gumbel-Softmax
• Convolutional Neural Networks for Sentence Classification
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Sequence Level Training with Recurrent Neural Networks
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Get To The Point: Summarization with Pointer-Generator Networks
• Unsupervised Neural Machine Translation
• Multiplicative LSTM for sequence modelling
• Language as a Latent Variable: Discrete Generative Models for Sentence Compression
• Unsupervised Cipher Cracking Using Discrete GANs

Cited by in corpus (8)

• Zero-Resource Knowledge-Grounded Dialogue Generation
• Hierarchical Transformers for Multi-Document Summarization
• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
• Leveraging Lead Bias for Zero-shot Abstractive News Summarization
• Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking
• Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
• A Comprehensive Attempt to Research Statement Generation
• Boosting Summarization with Normalizing Flows and Aggressive Training