Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models

Abstract

Sequence-to-sequence models have been applied to the conversation response generation problem where the source sequence is the conversation history and the target sequence is the response. Unlike translation, conversation responding is inherently creative. The generation of long, informative, coherent, and diverse responses remains a hard task. In this work, we focus on the single turn setting. We add self-attention to the decoder to maintain coherence in longer responses, and we propose a practical approach, called the glimpse-model, for scaling to large datasets. We introduce a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process. We trained on a combined data set of over 2.3B conversation messages mined from the web. In human evaluation studies, our method produces longer responses overall, with a higher proportion rated as acceptable and excellent as length increases, compared to baseline sequence-to-sequence models with explicit length-promotion. A back-off strategy produces better responses overall, in the full spectrum of lengths.

To appear in EMNLP 2017

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• A Neural Conversational Model
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Deep Reinforcement Learning for Dialogue Generation
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking

Cited by in corpus (13)

• Learning to Decode for Future Success
• Flexible and Scalable State Tracking Framework for Goal-Oriented Dialogue Systems
• OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts
• Neural Net Models for Open-Domain Discourse Coherence
• Generating Multiple Diverse Responses for Short-Text Conversation
• An Attention Mechanism for Answer Selection Using a Combined Global and Local View
• Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots
• OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts
• Netizen-Style Commenting on Fashion Photos: Dataset and Diversity Measures
• Affective Neural Response Generation
• Towards Controlled and Diverse Generation of Article Comments
• Grounding Conversations with Improvised Dialogues
• Augmenting Machine Learning with Information Retrieval to Recommend Real Cloned Code Methods for Code Completion