Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference

Abstract

Intent detection is one of the core components of goal-oriented dialog systems, and detecting out-of-scope (OOS) intents is also a practically important skill. Few-shot learning is attracting much attention to mitigate data scarcity, but OOS detection becomes even more challenging. In this paper, we present a simple yet effective approach, discriminative nearest neighbor classification with deep self-attention. Unlike softmax classifiers, we leverage BERT-style pairwise encoding to train a binary classifier that estimates the best matched training example for a user input. We propose to boost the discriminative ability by transferring a natural language inference (NLI) model. Our extensive experiments on a large-scale multi-domain intent detection task show that our method achieves more stable and accurate in-domain and OOS detection accuracy than RoBERTa-based classifiers and embedding-based nearest neighbor approaches. More notably, the NLI transfer enables our 10-shot model to perform competitively with 50-shot or even full-shot classifiers, while we can keep the inference time constant by leveraging a faster embedding retrieval model.

arXiv comment19 pages, accepted by EMNLP 2020 main conference as a long paper. Code will be available at https://github.com/salesforce/DNNC-few-shot-intent

References in corpus (12)

• Decoupled Weight Decay Regularization
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• On the difficulty of training Recurrent Neural Networks
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• XNLI: Evaluating Cross-lingual Sentence Representations
• Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation
• Low Resource Text Classification with ULMFit and Backtranslation
• CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
• Question Answering from Unstructured Text by Retrieval and Comprehension
• When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification

Cited by in corpus (9)

• Neural Data Augmentation via Example Extrapolation
• ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers
• A Joint and Domain-Adaptive Approach to Spoken Language Understanding
• Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training
• ConvFiT: Conversational Fine-Tuning of Pretrained Language Models
• Pseudo Siamese Network for Few-shot Intent Generation
• Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care
• Effectiveness of Pre-training for Few-shot Intent Classification