Memory-Augmented Recurrent Networks for Dialogue Coherence

Abstract

Recent dialogue approaches operate by reading each word in a conversation history, and aggregating accrued dialogue information into a single state. This fixed-size vector is not expandable and must maintain a consistent format over time. Other recent approaches exploit an attention mechanism to extract useful information from past conversational utterances, but this introduces an increased computational complexity. In this work, we explore the use of the Neural Turing Machine (NTM) to provide a more permanent and flexible storage mechanism for maintaining dialogue coherence. Specifically, we introduce two separate dialogue architectures based on this NTM design. The first design features a sequence-to-sequence architecture with two separate NTM modules, one for each participant in the conversation. The second memory architecture incorporates a single NTM module, which stores parallel context information for both speakers. This second design also replaces the sequence-to-sequence architecture with a neural language model, to allow for longer context of the NTM and greater understanding of the dialogue history. We report perplexity performance for both models, and compare them to existing baselines.

Honors project, 12 pages

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• A Neural Conversational Model
• End-To-End Memory Networks
• A Persona-Based Neural Conversation Model
• Memory Augmented Neural Networks with Wormhole Connections
• Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes
• MojiTalk: Generating Emotional Responses at Scale
• A Hierarchical Latent Structure for Variational Conversation Modeling
• Structured Memory for Neural Turing Machines
• Towards Universal Dialogue State Tracking