A Survey of Available Corpora for Building Data-Driven Dialogue Systems

Abstract

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective.

56 pages including references and appendix, 5 tables and 1 figure; Under review for the Dialogue & Discourse journal. Update: paper has been rewritten and now includes several new datasets

References in corpus (15)

• Sequence to Sequence Learning with Neural Networks
• Sequence Transduction with Recurrent Neural Networks
• On Using Monolingual Corpora in Neural Machine Translation
• Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• A Persona-Based Neural Conversation Model
• End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning
• Continuously Learning Neural Dialogue Management
• deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
• Improved Deep Learning Baselines for Ubuntu Corpus Dialogs
• Dialog-based Language Learning
• Strategic Dialogue Management via Deep Reinforcement Learning
• Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
• Natural Language Generation in Dialogue using Lexicalized and Delexicalized Data
• Personalizing a Dialogue System with Transfer Reinforcement Learning

Cited by in corpus (56)

• Deep Reinforcement Learning for Dialogue Generation
• Personalizing Dialogue Agents: I have a dog, do you have pets too?
• A Network-based End-to-End Trainable Task-oriented Dialogue System
• QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
• Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation
• DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue
• Mutual Information and Diverse Decoding Improve Neural Machine Translation
• Personalized Dialogue Generation with Diversified Traits
• Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset
• Learning End-to-End Goal-Oriented Dialog
• Personalization in Goal-Oriented Dialog
• Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
• Challenges in Building Intelligent Open-domain Dialog Systems
• Deep Learning Based Chatbot Models
• Recent advances in conversational NLP : Towards the standardization of Chatbot building
• Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
• On Evaluating and Comparing Open Domain Dialog Systems
• Edina: Building an Open Domain Socialbot with Self-dialogues
• Lifelong and Interactive Learning of Factual Knowledge in Dialogues
• GuessWhat?! Visual object discovery through multi-modal dialogue
• Getting To Know You: User Attribute Extraction from Dialogues
• Designing for Health Chatbots
• Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks
• A Wizard of Oz Study Simulating API Usage Dialogues with a Virtual Assistant
• Dialog state tracking, a machine reading approach using Memory Network
• Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation
• RMM: A Recursive Mental Model for Dialog Navigation
• Talking to myself: self-dialogues as data for conversational agents
• Lifelong Knowledge Learning in Rule-based Dialogue Systems
• HarperValleyBank: A Domain-Specific Spoken Dialog Corpus
• Towards Exploiting Background Knowledge for Building Conversation Systems
• A Dataset for Document Grounded Conversations
• A Hybrid Architecture for Multi-Party Conversational Systems
• When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications
• Extending Neural Generative Conversational Model using External Knowledge Sources
• Retrieval-Enhanced Adversarial Training for Neural Response Generation
• MOSS: End-to-End Dialog System Framework with Modular Supervision
• Why Do Neural Response Generation Models Prefer Universal Replies?
• A Large-Scale Chinese Short-Text Conversation Dataset
• Designing dialogue systems: A mean, grumpy, sarcastic chatbot in the browser
• Multi-task learning to improve natural language understanding
• A Review on Dyadic Conversation Visualizations - Purposes, Data, Lens of Analysis
• The Apiza Corpus: API Usage Dialogues with a Simulated Virtual Assistant
• Examining Cooperation in Visual Dialog Models
• An Annotated Corpus of Relational Strategies in Customer Service
• Multilingual Dialogue Generation with Shared-Private Memory
• Speaker Fluency Level Classification Using Machine Learning Techniques
• A Dataset for Building Code-Mixed Goal Oriented Conversation Systems
• A Bi-Encoder LSTM Model For Learning Unstructured Dialogs
• Achieving Fluency and Coherency in Task-oriented Dialog
• Predicting User Engagement Status for Online Evaluation of Intelligent Assistants
• Learning End-to-End Goal-Oriented Dialog with Multiple Answers
• Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics
• A question-answering system for aircraft pilots' documentation
• Building a Legal Dialogue System: Development Process, Challenges and Opportunities
• The RLLChatbot: a solution to the ConvAI challenge