Efficient Summarization with Read-Again and Copy Mechanism

Abstract

Encoder-decoder models have been widely used to solve sequence to sequence prediction tasks. However current approaches suffer from two shortcomings. First, the encoders compute a representation of each word taking into account only the history of the words it has read so far, yielding suboptimal representations. Second, current decoders utilize large vocabularies in order to minimize the problem of unknown words, resulting in slow decoding times. In this paper we address both shortcomings. Towards this goal, we first introduce a simple mechanism that first reads the input sequence before committing to a representation of each word. Furthermore, we propose a simple copy mechanism that is able to exploit very small vocabularies and handle out-of-vocabulary words. We demonstrate the effectiveness of our approach on the Gigaword dataset and DUC competition outperforming the state-of-the-art.

11 pages, 4 figures, 5 tables

References in corpus (6)

• Natural Language Processing (almost) from Scratch
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• Neural Summarization by Extracting Sentences and Words
• Addressing the Rare Word Problem in Neural Machine Translation
• Random Walk Initialization for Training Very Deep Feedforward Networks

Cited by in corpus (19)

• A Deep Reinforced Model for Abstractive Summarization
• Selective Encoding for Abstractive Sentence Summarization
• Get To The Point: Summarization with Pointer-Generator Networks
• Structure-Infused Copy Mechanisms for Abstractive Summarization
• Neural Abstractive Text Summarization with Sequence-to-Sequence Models
• Explanation as a Defense of Recommendation
• Topic Augmented Generator for Abstractive Summarization
• Abstractive Summarization Using Attentive Neural Techniques
• Summary Level Training of Sentence Rewriting for Abstractive Summarization
• Deep Keyphrase Generation
• Teaching Machines to Converse
• Closed-Book Training to Improve Summarization Encoder Memory
• Repurposing Decoder-Transformer Language Models for Abstractive Summarization
• Reinforced Generative Adversarial Network for Abstractive Text Summarization
• Context-Dependent Semantic Parsing over Temporally Structured Data
• Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization
• Improving Abstractive Text Summarization with History Aggregation
• Using Statistical and Semantic Models for Multi-Document Summarization
• Abstractive Summarization Improved by WordNet-based Extractive Sentences