Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System

Abstract

Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set. In the practical application, a reliable dialogue system should know what it does not know. In this paper, we introduce a new task, Novel Slot Detection (NSD), in the task-oriented dialogue system. NSD aims to discover unknown or out-of-domain slot types to strengthen the capability of a dialogue system based on in-domain training data. Besides, we construct two public NSD datasets, propose several strong NSD baselines, and establish a benchmark for future work. Finally, we conduct exhaustive experiments and qualitative analysis to comprehend key challenges and provide new guidance for future directions.

arXiv commentAccepted by ACL2021

References in corpus (4)

• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
• Transfer Learning for Sequence Labeling Using Source Model and Target Data
• Deep Unknown Intent Detection with Margin Loss