Schema-Guided Dialogue State Tracking Task at DSTC8
arXiv:2002.01359
Abstract
This paper gives an overview of the Schema-Guided Dialogue State Tracking task of the 8th Dialogue System Technology Challenge. The goal of this task is to develop dialogue state tracking models suitable for large-scale virtual assistants, with a focus on data-efficient joint modeling across domains and zero-shot generalization to new APIs. This task provided a new dataset consisting of over 16000 dialogues in the training set spanning 16 domains to highlight these challenges, and a baseline model capable of zero-shot generalization to new APIs. Twenty-five teams participated, developing a range of neural network models, exceeding the performance of the baseline model by a very high margin. The submissions incorporated a variety of pre-trained encoders and data augmentation techniques. This paper describes the task definition, dataset and evaluation methodology. We also summarize the approach and results of the submitted systems to highlight the overall trends in the state-of-the-art.
Presented at DSTC workshop, AAAI 2020. arXiv admin note: text overlap with arXiv:1909.05855
References in corpus (7)
- Text Chunking using Transformation-Based Learning
- Building a Conversational Agent Overnight with Dialogue Self-Play
- Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset
- An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide & Deep Classification
- Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking
- Goal-Oriented Multi-Task BERT-Based Dialogue State Tracker
- The SPPD System for Schema Guided Dialogue State Tracking Challenge
Cited by in corpus (5)
- Show, Don't Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue
- SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems
- Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems
- Attention Guided Dialogue State Tracking with Sparse Supervision
- AUGNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation