Adaptive Task Sampling for Meta-Learning
arXiv:2007.08735
Abstract
Meta-learning methods have been extensively studied and applied in computer vision, especially for few-shot classification tasks. The key idea of meta-learning for few-shot classification is to mimic the few-shot situations faced at test time by randomly sampling classes in meta-training data to construct few-shot tasks for episodic training. While a rich line of work focuses solely on how to extract meta-knowledge across tasks, we exploit the complementary problem on how to generate informative tasks. We argue that the randomly sampled tasks could be sub-optimal and uninformative (e.g., the task of classifying "dog" from "laptop" is often trivial) to the meta-learner. In this paper, we propose an adaptive task sampling method to improve the generalization performance. Unlike instance based sampling, task based sampling is much more challenging due to the implicit definition of the task in each episode. Therefore, we accordingly propose a greedy class-pair based sampling method, which selects difficult tasks according to class-pair potentials. We evaluate our adaptive task sampling method on two few-shot classification benchmarks, and it achieves consistent improvements across different feature backbones, meta-learning algorithms and datasets.
ECCV2020
References in corpus (18)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- A Simple Neural Attentive Meta-Learner
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- Meta-Learning for Semi-Supervised Few-Shot Classification
- Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
- On First-Order Meta-Learning Algorithms
- Online Batch Selection for Faster Training of Neural Networks
- Not All Samples Are Created Equal: Deep Learning with Importance Sampling
- Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling
- Variance Reduction in SGD by Distributed Importance Sampling
- How to train your MAML
- A Primer on PAC-Bayesian Learning
- Biased Importance Sampling for Deep Neural Network Training
- A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent
- Minimizing the Maximal Loss: How and Why?