Source-side Prediction for Neural Headline Generation

Abstract

The encoder-decoder model is widely used in natural language generation tasks. However, the model sometimes suffers from repeated redundant generation, misses important phrases, and includes irrelevant entities. Toward solving these problems we propose a novel source-side token prediction module. Our method jointly estimates the probability distributions over source and target vocabularies to capture a correspondence between source and target tokens. The experiments show that the proposed model outperforms the current state-of-the-art method in the headline generation task. Additionally, we show that our method has an ability to learn a reasonable token-wise correspondence without knowing any true alignments.

19 pages

References in corpus (8)

• Convolutional Sequence to Sequence Learning
• Effective Approaches to Attention-based Neural Machine Translation
• Selective Encoding for Abstractive Sentence Summarization
• Modeling Coverage for Neural Machine Translation
• Neural Machine Translation with Reconstruction
• Coverage Embedding Models for Neural Machine Translation
• Deep Recurrent Generative Decoder for Abstractive Text Summarization
• Neural Machine Translation with Word Predictions

Cited by in corpus (4)

• Multi-Task Learning for Cross-Lingual Abstractive Summarization
• This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation
• Direct Output Connection for a High-Rank Language Model
• Character n-gram Embeddings to Improve RNN Language Models