Incremental Transformer with Deliberation Decoder for Document Grounded Conversations

Abstract

Document Grounded Conversations is a task to generate dialogue responses when chatting about the content of a given document. Obviously, document knowledge plays a critical role in Document Grounded Conversations, while existing dialogue models do not exploit this kind of knowledge effectively enough. In this paper, we propose a novel Transformer-based architecture for multi-turn document grounded conversations. In particular, we devise an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents. Motivated by the human cognitive process, we design a two-pass decoder (Deliberation Decoder) to improve context coherence and knowledge correctness. Our empirical study on a real-world Document Grounded Dataset proves that responses generated by our model significantly outperform competitive baselines on both context coherence and knowledge relevance.

arXiv commentAccepted as a long paper at ACL 2019

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• A Neural Conversational Model
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• Modeling Coherence for Discourse Neural Machine Translation
• Story Ending Generation with Incremental Encoding and Commonsense Knowledge

Cited by in corpus (6)

• Bridging Text and Video: A Universal Multimodal Transformer for Video-Audio Scene-Aware Dialog
• Knowledge Injection into Dialogue Generation via Language Models
• RefNet: A Reference-aware Network for Background Based Conversation
• Knowledge-Grounded Dialogue Generation with Pre-trained Language Models
• Towards information-rich, logical text generation with knowledge-enhanced neural models
• Cross Copy Network for Dialogue Generation