Augmenting Transformers with KNN-Based Composite Memory for Dialogue

Abstract

Various machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable of storing this knowledge. We propose augmenting generative Transformer neural networks with KNN-based Information Fetching (KIF) modules. Each KIF module learns a read operation to access fixed external knowledge. We apply these modules to generative dialog modeling, a challenging task where information must be flexibly retrieved and incorporated to maintain the topic and flow of conversation. We demonstrate the effectiveness of our approach by identifying relevant knowledge required for knowledgeable but engaging dialog from Wikipedia, images, and human-written dialog utterances, and show that leveraging this retrieved information improves model performance, measured by automatic and human evaluation.

References in corpus (16)

• YFCC100M: The New Data in Multimedia Research
• End-To-End Memory Networks
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• REALM: Retrieval-Augmented Language Model Pre-Training
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
• Improving Neural Language Models with a Continuous Cache
• Reading Wikipedia to Answer Open-Domain Questions
• Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
• ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
• Generative Deep Neural Networks for Dialogue: A Short Review
• Large Memory Layers with Product Keys
• Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
• Hierarchical Memory Networks
• Augmenting Self-attention with Persistent Memory
• Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation

Cited by in corpus (9)

• Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey
• Nearest Neighbor Machine Translation
• GNN-LM: Language Modeling based on Global Contexts via GNN
• Current Limitations of Language Models: What You Need is Retrieval
• Cross-Domain Generalization Through Memorization: A Study of Nearest Neighbors in Neural Duplicate Question Detection
• Generating Related Work
• Local Knowledge Powered Conversational Agents
• Cross-Modal Retrieval Augmentation for Multi-Modal Classification
• Towards meaningful, grounded conversations with intelligent agents