A Repository of Conversational Datasets

Abstract

Progress in Machine Learning is often driven by the availability of large datasets, and consistent evaluation metrics for comparing modeling approaches. To this end, we present a repository of conversational datasets consisting of hundreds of millions of examples, and a standardised evaluation procedure for conversational response selection models using '1-of-100 accuracy'. The repository contains scripts that allow researchers to reproduce the standard datasets, or to adapt the pre-processing and data filtering steps to their needs. We introduce and evaluate several competitive baselines for conversational response selection, whose implementations are shared in the repository, as well as a neural encoder model that is trained on the entire training set.

References in corpus (9)

• A Neural Conversational Model
• Universal Sentence Encoder
• Efficient Natural Language Response Suggestion for Smart Reply
• Smart Reply: Automated Response Suggestion for Email
• Generative Deep Neural Networks for Dialogue: A Short Review
• Training Neural Response Selection for Task-Oriented Dialogue Systems
• Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
• Designing for Health Chatbots
• Customized Nonlinear Bandits for Online Response Selection in Neural Conversation Models

Cited by in corpus (3)

• Report from the NSF Future Directions Workshop, Toward User-Oriented Agents: Research Directions and Challenges
• Context Matters in Semantically Controlled Language Generation for Task-oriented Dialogue Systems
• Multi-turn Response Selection using Dialogue Dependency Relations