BanditSum: Extractive Summarization as a Contextual Bandit

Abstract

In this work, we propose a novel method for training neural networks to perform single-document extractive summarization without heuristically-generated extractive labels. We call our approach BanditSum as it treats extractive summarization as a contextual bandit (CB) problem, where the model receives a document to summarize (the context), and chooses a sequence of sentences to include in the summary (the action). A policy gradient reinforcement learning algorithm is used to train the model to select sequences of sentences that maximize ROUGE score. We perform a series of experiments demonstrating that BanditSum is able to achieve ROUGE scores that are better than or comparable to the state-of-the-art for extractive summarization, and converges using significantly fewer update steps than competing approaches. In addition, we show empirically that BanditSum performs significantly better than competing approaches when good summary sentences appear late in the source document.

12 pages, 2 figures, EMNLP 2018

References in corpus (4)

• Efficient Estimation of Word Representations in Vector Space
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Teaching Machines to Read and Comprehend
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents

Cited by in corpus (8)

• Learning to summarize from human feedback
• Investigating Entropy for Extractive Document Summarization
• AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization
• PoBRL: Optimizing Multi-Document Summarization by Blending Reinforcement Learning Policies
• BanditRank: Learning to Rank Using Contextual Bandits
• Fact-level Extractive Summarization with Hierarchical Graph Mask on BERT
• Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
• Boosting Image Recognition with Non-differentiable Constraints