Contextual Out-of-Domain Utterance Handling With Counterfeit Data Augmentation

Abstract

Neural dialog models often lack robustness to anomalous user input and produce inappropriate responses which leads to frustrating user experience. Although there are a set of prior approaches to out-of-domain (OOD) utterance detection, they share a few restrictions: they rely on OOD data or multiple sub-domains, and their OOD detection is context-independent which leads to suboptimal performance in a dialog. The goal of this paper is to propose a novel OOD detection method that does not require OOD data by utilizing counterfeit OOD turns in the context of a dialog. For the sake of fostering further research, we also release new dialog datasets which are 3 publicly available dialog corpora augmented with OOD turns in a controllable way. Our method outperforms state-of-the-art dialog models equipped with a conventional OOD detection mechanism by a large margin in the presence of OOD utterances.

arXiv commentICASSP 2019

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• Building a Conversational Agent Overnight with Dialogue Self-Play
• Learning End-to-End Goal-Oriented Dialog
• Neural Sentence Embedding using Only In-domain Sentences for Out-of-domain Sentence Detection in Dialog Systems

Cited by in corpus (4)

• Alquist 4.0: Towards Social Intelligence Using Generative Models and Dialogue Personalization
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
• Enhancing Out-Of-Domain Utterance Detection with Data Augmentation Based on Word Embeddings
• Data-Efficient Methods for Dialogue Systems