Coherent Dialogue with Attention-based Language Models

Abstract

We model coherent conversation continuation via RNN-based dialogue models equipped with a dynamic attention mechanism. Our attention-RNN language model dynamically increases the scope of attention on the history as the conversation continues, as opposed to standard attention (or alignment) models with a fixed input scope in a sequence-to-sequence model. This allows each generated word to be associated with the most relevant words in its corresponding conversation history. We evaluate the model on two popular dialogue datasets, the open-domain MovieTriples dataset and the closed-domain Ubuntu Troubleshoot dataset, and achieve significant improvements over the state-of-the-art and baselines on several metrics, including complementary diversity-based metrics, human evaluation, and qualitative visualizations. We also show that a vanilla RNN with dynamic attention outperforms more complex memory models (e.g., LSTM and GRU) by allowing for flexible, long-distance memory. We promote further coherence via topic modeling-based reranking.

To appear at AAAI 2017

Cited by in corpus (15)

• DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
• Modeling Multi-turn Conversation with Deep Utterance Aggregation
• Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models
• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
• Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
• Neural Language Generation: Formulation, Methods, and Evaluation
• Emotional Neural Language Generation Grounded in Situational Contexts
• Reflective Decoding Network for Image Captioning
• Joint Embedding Learning of Educational Knowledge Graphs
• The Rapidly Changing Landscape of Conversational Agents
• Multilingual Dialogue Generation with Shared-Private Memory
• Relevance-Promoting Language Model for Short-Text Conversation
• Addressee and Response Selection for Multilingual Conversation
• Contextual Biasing of Language Models for Speech Recognition in Goal-Oriented Conversational Agents
• SAM: Semantic Attribute Modulation for Language Modeling and Style Variation