Is Support Set Diversity Necessary for Meta-Learning?
arXiv:2011.14048
Abstract
Meta-learning is a popular framework for learning with limited data in which an algorithm is produced by training over multiple few-shot learning tasks. For classification problems, these tasks are typically constructed by sampling a small number of support and query examples from a subset of the classes. While conventional wisdom is that task diversity should improve the performance of meta-learning, in this work we find evidence to the contrary: we propose a modification to traditional meta-learning approaches in which we keep the support sets fixed across tasks, thus reducing task diversity. Surprisingly, we find that not only does this modification not result in adverse effects, it almost always improves the performance for a variety of datasets and meta-learning methods. We also provide several initial analyses to understand this phenomenon. Our work serves to: (i) more closely investigate the effect of support set construction for the problem of meta-learning, and (ii) suggest a simple, general, and competitive baseline for few-shot learning.
References in corpus (16)
- Prototypical Networks for Few-shot Learning
- Wide Residual Networks
- Theoretical Models of Learning to Learn
- Matching Networks for One Shot Learning
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
- Three Factors Influencing Minima in SGD
- Meta-Learning with Implicit Gradients
- Meta-Learning with Latent Embedding Optimization
- TADAM: Task dependent adaptive metric for improved few-shot learning
- A Closer Look at Few-shot Classification
- Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
- Meta-Learning without Memorization
- Leveraging the Feature Distribution in Transfer-based Few-Shot Learning
- Task-Robust Model-Agnostic Meta-Learning
- Task Augmentation by Rotating for Meta-Learning
- Sufficiently Accurate Model Learning