PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable
arXiv:1910.07931
Abstract
Pre-training models have been proved effective for a wide range of natural language processing tasks. Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations, including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation. We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation. Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network. Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.
Accepted for publication at ACL2020. First two authors contributed equally
References in corpus (6)
- Microsoft COCO Captions: Data Collection and Evaluation Server
- A Neural Conversational Model
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- Wizard of Wikipedia: Knowledge-Powered Conversational agents
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
- TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents