Discriminative Adversarial Search for Abstractive Summarization
arXiv:2002.10375
Abstract
We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS), which has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. Inspired by Generative Adversarial Networks (GANs), wherein a discriminator is used to improve the generator, our method differs from GANs in that the generator parameters are not updated at training time and the discriminator is only used to drive sequence generation at inference time. We investigate the effectiveness of the proposed approach on the task of Abstractive Summarization: the results obtained show that a naive application of DAS improves over the state-of-the-art methods, with further gains obtained via discriminator retraining. Moreover, we show how DAS can be effective for cross-domain adaptation. Finally, all results reported are obtained without additional rule-based filtering strategies, commonly used by the best performing systems available: this indicates that DAS can effectively be deployed without relying on post-hoc modifications of the generated outputs.
ICML 2020
References in corpus (15)
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Teaching Machines to Read and Comprehend
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- A Deep Reinforced Model for Abstractive Summarization
- Unified Language Model Pre-training for Natural Language Understanding and Generation
- Sequence Level Training with Recurrent Neural Networks
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- Professor Forcing: A New Algorithm for Training Recurrent Networks
- Get To The Point: Summarization with Pointer-Generator Networks
- Language GANs Falling Short
- Analyzing Uncertainty in Neural Machine Translation
- Bridging the Gap between Training and Inference for Neural Machine Translation
- PyText: A Seamless Path from NLP research to production
- Answers Unite! Unsupervised Metrics for Reinforced Summarization Models
- Adding A Filter Based on The Discriminator to Improve Unconditional Text Generation