Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks

Abstract

Meta-learning algorithms produce feature extractors which achieve state-of-the-art performance on few-shot classification. While the literature is rich with meta-learning methods, little is known about why the resulting feature extractors perform so well. We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models which are trained classically. In doing so, we introduce and verify several hypotheses for why meta-learned models perform better. Furthermore, we develop a regularizer which boosts the performance of standard training routines for few-shot classification. In many cases, our routine outperforms meta-learning while simultaneously running an order of magnitude faster.

arXiv commentICML 2020

References in corpus (6)

• Prototypical Networks for Few-shot Learning
• Similarity of Neural Network Representations Revisited
• A Closer Look at Few-shot Classification
• Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
• Understanding Generalization through Visualizations
• Generalized Adaptation for Few-Shot Learning

Cited by in corpus (9)

• Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
• Meta-learning an Intermediate Representation for Few-shot Block-wise Prediction of Landslide Susceptibility
• Data Augmentation for Meta-Learning
• Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
• How Important is the Train-Validation Split in Meta-Learning?
• How Does the Task Landscape Affect MAML Performance?
• Why Do Better Loss Functions Lead to Less Transferable Features?
• Meta-learning Transferable Representations with a Single Target Domain
• Towards Enabling Meta-Learning from Target Models