Hierarchical Recurrent Attention Network for Response Generation

Abstract

We study multi-turn response generation in chatbots where a response is generated according to a conversation context. Existing work has modeled the hierarchy of the context, but does not pay enough attention to the fact that words and utterances in the context are differentially important. As a result, they may lose important information in context and generate irrelevant responses. We propose a hierarchical recurrent attention network (HRAN) to model both aspects in a unified framework. In HRAN, a hierarchical attention mechanism attends to important parts within and among utterances with word level attention and utterance level attention respectively. With the word level attention, hidden vectors of a word level encoder are synthesized as utterance vectors and fed to an utterance level encoder to construct hidden representations of the context. The hidden vectors of the context are then processed by the utterance level attention and formed as context vectors for decoding the response. Empirical studies on both automatic evaluation and human judgment show that HRAN can significantly outperform state-of-the-art models for multi-turn response generation.

Cited by in corpus (9)

• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
• Designing for Health Chatbots
• ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation
• Neural Response Generation with Meta-Words
• Self-Supervised Dialogue Learning
• Recovering Dropped Pronouns in Chinese Conversations via Modeling Their Referents
• Modeling Semantic Relationship in Multi-turn Conversations with Hierarchical Latent Variables
• Listening between the Lines: Learning Personal Attributes from Conversations
• A Persona-based Multi-turn Conversation Model in an Adversarial Learning Framework