Learning to Retrieve Entity-Aware Knowledge and Generate Responses with Copy Mechanism for Task-Oriented Dialogue Systems

Abstract

Task-oriented conversational modeling with unstructured knowledge access, as track 1 of the 9th Dialogue System Technology Challenges (DSTC 9), requests to build a system to generate response given dialogue history and knowledge access. This challenge can be separated into three subtasks, (1) knowledge-seeking turn detection, (2) knowledge selection, and (3) knowledge-grounded response generation. We use pre-trained language models, ELECTRA and RoBERTa, as our base encoder for different subtasks. For subtask 1 and 2, the coarse-grained information like domain and entity are used to enhance knowledge usage. For subtask 3, we use a latent variable to encode dialog history and selected knowledge better and generate responses combined with copy mechanism. Meanwhile, some useful post-processing strategies are performed on the model's final output to make further knowledge usage in the generation task. As shown in released evaluation results, our proposed system ranks second under objective metrics and ranks fourth under human metrics.

arXiv commentAccepted by AAAI 2021, Workshop on DSTC 9

References in corpus (8)

• Auto-Encoding Variational Bayes
• Gaussian Error Linear Units (GELUs)
• A Neural Conversational Model
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
• Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models
• PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
• A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots