Few-shot Text Classification with Distributional Signatures

Abstract

In this paper, we explore meta-learning for few-shot text classification. Meta-learning has shown strong performance in computer vision, where low-level patterns are transferable across learning tasks. However, directly applying this approach to text is challenging--lexical features highly informative for one task may be insignificant for another. Thus, rather than learning solely from words, our model also leverages their distributional signatures, which encode pertinent word occurrence patterns. Our model is trained within a meta-learning framework to map these signatures into attention scores, which are then used to weight the lexical representations of words. We demonstrate that our model consistently outperforms prototypical networks learned on lexical knowledge (Snell et al., 2017) in both few-shot text classification and relation classification by a significant margin across six benchmark datasets (20.0% on average in 1-shot classification).

arXiv commentICLR 2020

References in corpus (6)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• A Structured Self-attentive Sentence Embedding
• Few-Shot Learning with Graph Neural Networks
• Learning to Transfer

Cited by in corpus (21)

• Making Pre-trained Language Models Better Few-shot Learners
• A Survey on Text Classification: From Shallow to Deep Learning
• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners
• Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
• Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning
• FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
• When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications
• Dynamic Memory Induction Networks for Few-Shot Text Classification
• FLEX: Unifying Evaluation for Few-Shot NLP
• Few-shot Sequence Learning with Transformers
• Knowledge Guided Metric Learning for Few-Shot Text Classification
• Few-shot Conformal Prediction with Auxiliary Tasks
• ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection
• Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
• Learning Stable Classifiers by Transferring Unstable Features
• Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes
• Prior Omission of Dissimilar Source Domain(s) for Cost-Effective Few-Shot Learning
• Revisiting Self-Training for Few-Shot Learning of Language Model
• An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling