Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems

Abstract

Existing goal-oriented dialogue datasets focus mainly on identifying slots and values. However, customer support interactions in reality often involve agents following multi-step procedures derived from explicitly-defined company policies as well. To study customer service dialogue systems in more realistic settings, we introduce the Action-Based Conversations Dataset (ABCD), a fully-labeled dataset with over 10K human-to-human dialogues containing 55 distinct user intents requiring unique sequences of actions constrained by policies to achieve task success. We propose two additional dialog tasks, Action State Tracking and Cascading Dialogue Success, and establish a series of baselines involving large-scale, pre-trained language models on this dataset. Empirical results demonstrate that while more sophisticated networks outperform simpler models, a considerable gap (50.8% absolute accuracy) still exists to reach human-level performance on ABCD.

16 pages, 5 figures. Accepted at NAACL 2021

References in corpus (5)

• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
• Building a Conversational Agent Overnight with Dialogue Self-Play
• Training Neural Response Selection for Task-Oriented Dialogue Systems
• Schema-Guided Dialogue State Tracking Task at DSTC8
• An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide & Deep Classification