Discourse-Aware Neural Extractive Text Summarization

Abstract

Recently BERT has been adopted for document encoding in state-of-the-art text summarization models. However, sentence-based extractive models often result in redundant or uninformative phrases in the extracted summaries. Also, long-range dependencies throughout a document are not well captured by BERT, which is pre-trained on sentence pairs instead of documents. To address these issues, we present a discourse-aware neural summarization model - DiscoBert. DiscoBert extracts sub-sentential discourse units (instead of sentences) as candidates for extractive selection on a finer granularity. To capture the long-range dependencies among discourse units, structural discourse graphs are constructed based on RST trees and coreference mentions, encoded with Graph Convolutional Networks. Experiments show that the proposed model outperforms state-of-the-art methods by a significant margin on popular summarization benchmarks compared to other BERT-base models.

arXiv commentTo appear at ACL 2020; Code available at https://github.com/jiacheng-xu/DiscoBERT

References in corpus (6)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Teaching Machines to Read and Comprehend
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents
• Faithful to the Original: Fact Aware Neural Abstractive Summarization
• Structured Neural Summarization
• AttSum: Joint Learning of Focusing and Summarization with Neural Attention

Cited by in corpus (6)

• Heterogeneous Graph Neural Networks for Extractive Document Summarization
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
• Extractive Summarization as Text Matching
• Identifying Introductions in Podcast Episodes from Automatically Generated Transcripts
• Centrality Meets Centroid: A Graph-based Approach for Unsupervised Document Summarization
• Hierarchical Summarization for Longform Spoken Dialog