Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning

Abstract

In few-shot learning scenarios, the challenge is to generalize and perform well on new unseen examples when only very few labeled examples are available for each task. Model-agnostic meta-learning (MAML) has gained the popularity as one of the representative few-shot learning methods for its flexibility and applicability to diverse problems. However, MAML and its variants often resort to a simple loss function without any auxiliary loss function or regularization terms that can help achieve better generalization. The problem lies in that each application and task may require different auxiliary loss function, especially when tasks are diverse and distinct. Instead of attempting to hand-design an auxiliary loss function for each application and task, we introduce a new meta-learning framework with a loss function that adapts to each task. Our proposed framework, named Meta-Learning with Task-Adaptive Loss Function (MeTAL), demonstrates the effectiveness and the flexibility across various domains, such as few-shot classification and few-shot regression.

ICCV 2021 (Oral). Code at https://github.com/baiksung/MeTAL

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• TADAM: Task dependent adaptive metric for improved few-shot learning
• Empirical Bayes Transductive Meta-Learning with Synthetic Gradients
• BOIL: Towards Representation Change for Few-shot Learning
• Learning to learn via Self-Critique
• Meta Dropout: Learning to Perturb Features for Generalization
• Meta-Learning with Neural Tangent Kernels