A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)

Abstract

The recent advance in neural network architecture and training algorithms have shown the effectiveness of representation learning. The neural network-based models generate better representation than the traditional ones. They have the ability to automatically learn the distributed representation for sentences and documents. To this end, we proposed a novel model that addresses several issues that are not adequately modeled by the previously proposed models, such as the memory problem and incorporating the knowledge of document structure. Our model uses a hierarchical structured self-attention mechanism to create the sentence and document embeddings. This architecture mirrors the hierarchical structure of the document and in turn enables us to obtain better feature representation. The attention mechanism provides extra source of information to guide the summary extraction. The new model treated the summarization task as a classification problem in which the model computes the respective probabilities of sentence-summary membership. The model predictions are broken up by several features such as information content, salience, novelty and positional representation. The proposed model was evaluated on two well-known datasets, the CNN / Daily Mail, and DUC 2002. The experimental results show that our model outperforms the current extractive state-of-the-art by a considerable margin.

arXiv comment8 pages, 4 figures, 2 tables, IEEE Access, pp. 1-1, 2018

References in corpus (10)

• Efficient Estimation of Word Representations in Vector Space
• Neural Machine Translation by Jointly Learning to Align and Translate
• ADADELTA: An Adaptive Learning Rate Method
• Teaching Machines to Read and Comprehend
• A Structured Self-attentive Sentence Embedding
• A Deep Reinforced Model for Abstractive Summarization
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• AttSum: Joint Learning of Focusing and Summarization with Neural Attention
• Classify or Select: Neural Architectures for Extractive Document Summarization

Cited by in corpus (9)

• Investigating Entropy for Extractive Document Summarization
• Multi-head Sequence Tagging Model for Grammatical Error Correction
• Self-Supervised Learning for Contextualized Extractive Summarization
• Does Head Label Help for Long-Tailed Multi-Label Text Classification
• Latent Semantic Analysis Approach for Document Summarization Based on Word Embeddings
• Extractive Research Slide Generation Using Windowed Labeling Ranking
• Towards Personalized and Human-in-the-Loop Document Summarization
• Detect and Classify -- Joint Span Detection and Classification for Health Outcomes
• Extractive Summarizer for Scholarly Articles