A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data

Abstract

Endowing dialogue systems with personas is essential to deliver more human-like conversations. However, this problem is still far from well explored due to the difficulties of both embodying personalities in natural languages and the persona sparsity issue observed in most dialogue corpora. This paper proposes a pre-training based personalized dialogue model that can generate coherent responses using persona-sparse dialogue data. In this method, a pre-trained language model is used to initialize an encoder and decoder, and personal attribute embeddings are devised to model richer dialogue contexts by encoding speakers' personas together with dialogue histories. Further, to incorporate the target persona in the decoding process and to balance its contribution, an attention routing structure is devised in the decoder to merge features extracted from the target persona and dialogue contexts using dynamically predicted weights. Our model can utilize persona-sparse dialogues in a unified manner during the training process, and can also control the amount of persona-related features to exhibit during the inference process. Both automatic and manual evaluation demonstrates that the proposed model outperforms state-of-the-art methods for generating more coherent and persona consistent responses with persona-sparse data.

Long paper accepted at AAAI 2020

References in corpus (7)

• ERNIE: Enhanced Representation through Knowledge Integration
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Personalizing Dialogue Agents: I have a dog, do you have pets too?
• Personalized Dialogue Generation with Diversified Traits
• Challenges in Building Intelligent Open-domain Dialog Systems
• Assigning personality/identity to a chatting machine for coherent conversation generation
• Training Millions of Personalized Dialogue Agents

Cited by in corpus (7)

• Personalized Dialogue Generation with Persona-Adaptive Attention
• An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation
• Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation
• Open-Domain Dialogue Generation Based on Pre-trained Language Models
• Pretrained Language Models for Dialogue Generation with Multiple Input Sources
• A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation
• StyleDGPT: Stylized Response Generation with Pre-trained Language Models