CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection

Abstract

In this paper, we formulate a more realistic and difficult problem setup for the intent detection task in natural language understanding, namely Generalized Few-Shot Intent Detection (GFSID). GFSID aims to discriminate a joint label space consisting of both existing intents which have enough labeled data and novel intents which only have a few examples for each class. To approach this problem, we propose a novel model, Conditional Text Generation with BERT (CG-BERT). CG-BERT effectively leverages a large pre-trained language model to generate text conditioned on the intent label. By modeling the utterance distribution with variational inference, CG-BERT can generate diverse utterances for the novel intents even with only a few utterances available. Experimental results show that CG-BERT achieves state-of-the-art performance on the GFSID task with 1-shot and 5-shot settings on two real-world datasets.

11 pages, 4 figures

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• SMOTE: Synthetic Minority Over-sampling Technique
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Prototypical Networks for Few-shot Learning
• Unified Language Model Pre-training for Natural Language Understanding and Generation
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
• Adversarial Generation of Natural Language
• Joint Slot Filling and Intent Detection via Capsule Neural Networks
• Relational Generalized Few-Shot Learning
• Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning
• When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification

Cited by in corpus (9)

• Meta-learning for Few-shot Natural Language Processing: A Survey
• Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions
• Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference
• Composed Variational Natural Language Generation for Few-shot Intents
• Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System
• GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
• Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning
• Joint Text and Label Generation for Spoken Language Understanding
• Pseudo Siamese Network for Few-shot Intent Generation