Schema-Guided Dialogue State Tracking Task at DSTC8

Abstract

This paper gives an overview of the Schema-Guided Dialogue State Tracking task of the 8th Dialogue System Technology Challenge. The goal of this task is to develop dialogue state tracking models suitable for large-scale virtual assistants, with a focus on data-efficient joint modeling across domains and zero-shot generalization to new APIs. This task provided a new dataset consisting of over 16000 dialogues in the training set spanning 16 domains to highlight these challenges, and a baseline model capable of zero-shot generalization to new APIs. Twenty-five teams participated, developing a range of neural network models, exceeding the performance of the baseline model by a very high margin. The submissions incorporated a variety of pre-trained encoders and data augmentation techniques. This paper describes the task definition, dataset and evaluation methodology. We also summarize the approach and results of the submitted systems to highlight the overall trends in the state-of-the-art.

Presented at DSTC workshop, AAAI 2020. arXiv admin note: text overlap with arXiv:1909.05855

References in corpus (4)

• Building a Conversational Agent Overnight with Dialogue Self-Play
• An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide & Deep Classification
• Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking
• Goal-Oriented Multi-Task BERT-Based Dialogue State Tracker