Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Abstract

Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approximate it. In particular, we propose a self-play scenario where the dialog system talks to itself and we calculate a combination of proxies such as sentiment and semantic coherence on the conversation trajectory. We show that this metric is capable of capturing the human-rated quality of a dialog model better than any automated metric known to-date, achieving a significant Pearson correlation (r>.7, p<.05). To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and human evaluation of static conversations, we perform extended experiments with a set of models, including several that make novel improvements to recent hierarchical dialog generation architectures through sentiment and semantic knowledge distillation on the utterance level. Finally, we open-source the interactive evaluation platform we built and the dataset we collected to allow researchers to efficiently deploy and evaluate dialog models.

arXiv comment33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• Deep Reinforcement Learning for Dialogue Generation
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• Topic Aware Neural Response Generation
• A Deep Reinforcement Learning Chatbot
• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• Unifying Human and Statistical Evaluation for Natural Language Generation
• On Evaluating and Comparing Open Domain Dialog Systems
• A Hierarchical Latent Structure for Variational Conversation Modeling

Cited by in corpus (13)

• A Literature Survey of Recent Advances in Chatbots
• Advances and Challenges in Conversational Recommender Systems: A Survey
• Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog
• Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytelling
• ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
• OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts
• Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models
• Probing Neural Dialog Models for Conversational Understanding
• Nearly Horizon-Free Offline Reinforcement Learning
• Discovering Dialog Structure Graph for Open-Domain Dialog Generation
• Designing Precise and Robust Dialogue Response Evaluators
• Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach
• Self-play for Data Efficient Language Acquisition