Induction Networks for Few-Shot Text Classification

Abstract

Text classification tends to struggle when data is deficient or when it needs to adapt to unseen classes. In such challenging scenarios, recent studies have used meta-learning to simulate the few-shot task, in which new queries are compared to a small support set at the sample-wise level. However, this sample-wise comparison may be severely disturbed by the various expressions in the same class. Therefore, we should be able to learn a general representation of each class in the support set and then compare it to new queries. In this paper, we propose a novel Induction Network to learn such a generalized class-wise representation, by innovatively leveraging the dynamic routing algorithm in meta-learning. In this way, we find the model is able to induce and generalize better. We evaluate the proposed model on a well-studied sentiment classification dataset (English) and a real-world dialogue intent classification dataset (Chinese). Experiment results show that on both datasets, the proposed model significantly outperforms the existing state-of-the-art approaches, proving the effectiveness of class-wise generalization in few-shot text classification.

7 pages, 3 figures

References in corpus (9)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• A Structured Self-attentive Sentence Embedding
• Unsupervised Domain Adaptation with Residual Transfer Networks
• A Simple Neural Attentive Meta-Learner
• Few-Shot Learning with Graph Neural Networks
• A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
• Lifelong Domain Word Embedding via Meta-Learning
• Action Recognition with Coarse-to-Fine Deep Feature Integration and Asynchronous Fusion

Cited by in corpus (5)

• When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification
• Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
• Linguistically-Enriched and Context-Aware Zero-shot Slot Filling
• Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition
• Designing Neural Speaker Embeddings with Meta Learning