Global Encoding for Abstractive Summarization

Abstract

In neural abstractive summarization, the conventional sequence-to-sequence (seq2seq) model often suffers from repetition and semantic irrelevance. To tackle the problem, we propose a global encoding framework, which controls the information flow from the encoder to the decoder based on the global information of the source context. It consists of a convolutional gated unit to perform global encoding to improve the representations of the source-side information. Evaluations on the LCSTS and the English Gigaword both demonstrate that our model outperforms the baseline models, and the analysis shows that our model is capable of reducing repetition.

Accepted by ACL 2018

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Effective Approaches to Attention-based Neural Machine Translation
• Rethinking the Inception Architecture for Computer Vision
• Selective Encoding for Abstractive Sentence Summarization
• Incorporating Copying Mechanism in Sequence-to-Sequence Learning
• Deep Recurrent Generative Decoder for Abstractive Text Summarization
• Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation
• Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation

Cited by in corpus (13)

• DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
• SGM: Sequence Generation Model for Multi-label Classification
• Neural Language Generation: Formulation, Methods, and Evaluation
• PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
• Deconvolution-Based Global Decoding for Neural Machine Translation
• Meaningful Answer Generation of E-Commerce Question-Answering
• ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
• An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
• Denoising based Sequence-to-Sequence Pre-training for Text Generation
• Deep Neural Network for Semantic-based Text Recognition in Images
• Abstractive Text Summarization by Incorporating Reader Comments
• Towards Controlled and Diverse Generation of Article Comments
• GRET: Global Representation Enhanced Transformer