Meta-learning for Few-shot Natural Language Processing: A Survey

Abstract

Few-shot natural language processing (NLP) refers to NLP tasks that are accompanied with merely a handful of labeled examples. This is a real-world challenge that an AI system must learn to handle. Usually we rely on collecting more auxiliary information or developing a more efficient learning algorithm. However, the general gradient-based optimization in high capacity models, if training from scratch, requires many parameter-updating steps over a large number of labeled examples to perform well (Snell et al., 2017). If the target task itself cannot provide more information, how about collecting more tasks equipped with rich annotations to help the model learning? The goal of meta-learning is to train a model on a variety of tasks with rich annotations, such that it can solve a new task using only a few labeled samples. The key idea is to train the model's initial parameters such that the model has maximal performance on a new task after the parameters have been updated through zero or a couple of gradient steps. There are already some surveys for meta-learning, such as (Vilalta and Drissi, 2002; Vanschoren, 2018; Hospedales et al., 2020). Nevertheless, this paper focuses on NLP domain, especially few-shot applications. We try to provide clearer definitions, progress summary and some common datasets of applying meta-learning to few-shot NLP.

no submission intent

References in corpus (7)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• On First-Order Meta-Learning Algorithms
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
• Meta-Learning: A Survey
• CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
• Knowledge Guided Metric Learning for Few-Shot Text Classification

Cited by in corpus (16)

• Trans4E: Link Prediction on Scholarly Knowledge Graphs
• ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users
• Mask-guided BERT for Few Shot Text Classification
• Grounded Language Learning Fast and Slow
• A Comparative Review of Recent Few-Shot Object Detection Algorithms
• AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization
• MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging
• Learning from similarity and information extraction from structured documents
• Composed Variational Natural Language Generation for Few-shot Intents
• FLEX: Unifying Evaluation for Few-Shot NLP
• Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start
• Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
• Faster Optimization-Based Meta-Learning Adaptation Phase
• Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections
• A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters
• Unsupervised Meta Learning for One Shot Title Compression in Voice Commerce