Hierarchical Recurrent Attention Network for Response Generation

Abstract

We study multi-turn response generation in chatbots where a response is generated according to a conversation context. Existing work has modeled the hierarchy of the context, but does not pay enough attention to the fact that words and utterances in the context are differentially important. As a result, they may lose important information in context and generate irrelevant responses. We propose a hierarchical recurrent attention network (HRAN) to model both aspects in a unified framework. In HRAN, a hierarchical attention mechanism attends to important parts within and among utterances with word level attention and utterance level attention respectively. With the word level attention, hidden vectors of a word level encoder are synthesized as utterance vectors and fed to an utterance level encoder to construct hidden representations of the context. The hidden vectors of the context are then processed by the utterance level attention and formed as context vectors for decoding the response. Empirical studies on both automatic evaluation and human judgment show that HRAN can significantly outperform state-of-the-art models for multi-turn response generation.

Cited by in corpus (33)

• Hierarchical Learning for Generation with Long Source Sequences
• Deep Learning Based Chatbot Models
• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
• Designing for Health Chatbots
• Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks
• Hierarchical Attention Networks for Medical Image Segmentation
• ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation
• Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?
• A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking
• Recovering Dropped Pronouns in Chinese Conversations via Modeling Their Referents
• Self-Supervised Dialogue Learning
• Conditional Text Generation for Harmonious Human-Machine Interaction
• Neural Response Generation with Meta-Words
• Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots
• The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection
• Regularizing Dialogue Generation by Imitating Implicit Scenarios
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots
• Modeling Semantic Relationship in Multi-turn Conversations with Hierarchical Latent Variables
• Understanding Chat Messages for Sticker Recommendation in Messaging Apps
• Group-wise Contrastive Learning for Neural Dialogue Generation
• Learning to Select Context in a Hierarchical and Global Perspective for Open-domain Dialogue Generation
• A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation
• Modeling Topical Relevance for Multi-Turn Dialogue Generation
• Modeling Psychotherapy Dialogues with Kernelized Hashcode Representations: A Nonparametric Information-Theoretic Approach
• Know Deeper: Knowledge-Conversation Cyclic Utilization Mechanism for Open-domain Dialogue Generation
• Listening between the Lines: Learning Personal Attributes from Conversations
• A Multi-Turn Emotionally Engaging Dialog Model
• A Persona-based Multi-turn Conversation Model in an Adversarial Learning Framework
• Unsupervised Context Rewriting for Open Domain Conversation
• GoChat: Goal-oriented Chatbots with Hierarchical Reinforcement Learning
• GTM: A Generative Triple-Wise Model for Conversational Question Generation
• HCNet: Hierarchical Context Network for Semantic Segmentation
• ScriptWriter: Narrative-Guided Script Generation