Hierarchical Transformers for Multi-Document Summarization

Abstract

In this paper, we develop a neural summarization model which can effectively process multiple input documents and distill Transformer architecture with the ability to encode documents in a hierarchical manner. We represent cross-document relationships via an attention mechanism which allows to share information as opposed to simply concatenating text spans and processing them as a flat sequence. Our model learns latent dependencies among textual units, but can also take advantage of explicit graph representations focusing on similarity or discourse relations. Empirical results on the WikiSum dataset demonstrate that the proposed architecture brings substantial improvements over several strong baselines.

to appear at ACL 2019

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Teaching Machines to Read and Comprehend
• A Deep Reinforced Model for Abstractive Summarization
• Structured Attention Networks
• MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization

Cited by in corpus (10)

• Pretrained Transformers for Text Ranking: BERT and Beyond
• DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media
• Recursively Summarizing Books with Human Feedback
• BP-Transformer: Modelling Long-Range Context via Binary Partitioning
• Learning to Encode Position for Transformer with Continuous Dynamical Model
• PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models
• User-Inspired Posterior Network for Recommendation Reason Generation
• A Comprehensive Attempt to Research Statement Generation
• Hierarchical GPT with Congruent Transformers for Multi-Sentence Language Models
• Injecting Hierarchy with U-Net Transformers