Empathetic Dialogue Generation with Pre-trained RoBERTa-GPT2 and External Knowledge
arXiv:2109.03004
Abstract
One challenge for dialogue agents is to recognize feelings of the conversation partner and respond accordingly. In this work, RoBERTa-GPT2 is proposed for empathetic dialogue generation, where the pre-trained auto-encoding RoBERTa is utilised as encoder and the pre-trained auto-regressive GPT-2 as decoder. With the combination of the pre-trained RoBERTa and GPT-2, our model realizes a new state-of-the-art emotion accuracy. To enable the empathetic ability of RoBERTa-GPT2 model, we propose a commonsense knowledge and emotional concepts extractor, in which the commonsensible and emotional concepts of dialogue context are extracted for the GPT-2 decoder. The experiment results demonstrate that the empathetic dialogue generation benefits from both pre-trained encoder-decoder architecture and external knowledge.
arXiv commentaccepted at International Workshop on Spoken Dialog System Technology (IWSDS) 2021
References in corpus (6)
- Sequence to Sequence Learning with Neural Networks
- NLTK: The Natural Language Toolkit
- ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
- TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
- EmotionLines: An Emotion Corpus of Multi-Party Conversations
- Generative Deep Neural Networks for Dialogue: A Short Review