Towards a Human-like Open-Domain Chatbot
arXiv:2001.09977
Abstract
We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We also propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of a human-like multi-turn conversation. Our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72% on multi-turn evaluation) suggests that a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher in absolute SSA than the existing chatbots we evaluated.
38 pages, 12 figures
References in corpus (5)
Cited by in corpus (13)
- The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
- DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue
- Controlling Style in Generated Dialogue
- Few-shot Natural Language Generation for Task-Oriented Dialog
- Robust Conversational AI with Grounded Text Generation
- MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
- Non-Autoregressive Neural Dialogue Generation
- Emora: An Inquisitive Social Chatbot Who Cares For You
- CareCall: a Call-Based Active Monitoring Dialog Agent for Managing COVID-19 Pandemic
- Dialogue Response Ranking Training with Large-Scale Human Feedback Data
- On the Discrepancy between Density Estimation and Sequence Generation
- StyleDGPT: Stylized Response Generation with Pre-trained Language Models
- Improvement of a dedicated model for open domain persona-aware dialogue generation