Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators

Abstract

Encoder-decoder based neural architectures serve as the basis of state-of-the-art approaches in end-to-end open domain dialog systems. Since most of such systems are trained with a maximum likelihood~(MLE) objective they suffer from issues such as lack of generalizability and the generic response problem, i.e., a system response that can be an answer to a large number of user utterances, e.g., "Maybe, I don't know." Having explicit feedback on the relevance and interestingness of a system response at each turn can be a useful signal for mitigating such issues and improving system quality by selecting responses from different approaches. Towards this goal, we present a system that evaluates chatbot responses at each dialog turn for coherence and engagement. Our system provides explicit turn-level dialog quality feedback, which we show to be highly correlated with human evaluation. To show that incorporating this feedback in the neural response generation models improves dialog quality, we present two different and complementary mechanisms to incorporate explicit feedback into a neural response generation model: reranking and direct modification of the loss function during training. Our studies show that a response generation model that incorporates these combined feedback mechanisms produce more engaging and coherent responses in an open-domain spoken dialog setting, significantly improving the response quality using both automatic and human evaluation.

References in corpus (16)

• Sequence to Sequence Learning with Neural Networks
• Effective Approaches to Attention-based Neural Machine Translation
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• An efficient framework for learning sentence representations
• Enriching Word Vectors with Subword Information
• Deep Reinforcement Learning for Dialogue Generation
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• Topic Aware Neural Response Generation
• Conversational AI: The Science Behind the Alexa Prize
• A Deep Reinforcement Learning Chatbot
• A Persona-Based Neural Conversation Model
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Topic-based Evaluation for Conversational Bots
• An Attentional Neural Conversation Model with Improved Specificity
• On Evaluating and Comparing Open Domain Dialog Systems
• Data Distillation for Controlling Specificity in Dialogue Generation

Cited by in corpus (9)

• Fine-Tuning Language Models from Human Preferences
• Recursively Summarizing Books with Human Feedback
• Learning to summarize from human feedback
• A Modular Task-oriented Dialogue System Using a Neural Mixture-of-Experts
• Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation
• Speech Sentiment and Customer Satisfaction Estimation in Socialbot Conversations
• Dynamic Knowledge Routing Network For Target-Guided Open-Domain Conversation
• HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations
• Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue Systems