Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models
arXiv:1910.07117
Abstract
In this work, we study how the finetuning stage in the pretrain-finetune framework changes the behavior of a pretrained neural language generator. We focus on the transformer encoder-decoder model for the open-domain dialogue response generation task. Our major finding is that after standard finetuning, the model forgets some of the important language generation skills acquired during large-scale pretraining. We demonstrate the forgetting phenomenon through a set of detailed behavior analysis from the perspectives of knowledge transfer, context sensitivity, and function space projection. As a preliminary attempt to alleviate the forgetting problem, we propose an intuitive finetuning strategy named "mix-review". We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent. Finally, we discuss interesting behavior of the resulting dialogue model and its implications.
References in corpus (13)
- Sequence to Sequence Learning with Neural Networks
- Overcoming catastrophic forgetting in neural networks
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- XLNet: Generalized Autoregressive Pretraining for Language Understanding
- Recent Trends in Deep Learning Based Natural Language Processing
- Skip-Thought Vectors
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
- Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks
- Real or Fake? Learning to Discriminate Machine from Human Generated Text
- Detecting egregious responses in neural sequence-to-sequence models
- Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning
- Representation Stability as a Regularizer for Improved Text Analytics Transfer Learning