Self-training for Few-shot Transfer Across Extreme Task Differences
arXiv:2010.07734
Abstract
Most few-shot learning techniques are pre-trained on a large, labeled "base dataset". In problem domains where such large labeled datasets are not available for pre-training (e.g., X-ray, satellite images), one must resort to pre-training in a different "source" problem domain (e.g., ImageNet), which can be very different from the desired target task. Traditional few-shot and transfer learning techniques fail in the presence of such extreme differences between the source and target tasks. In this paper, we present a simple and effective solution to tackle this extreme domain gap: self-training a source domain representation on unlabeled data from the target domain. We show that this improves one-shot performance on the target domain by 2.9 points on average on the challenging BSCD-FSL benchmark consisting of datasets from multiple domains. Our code is available at https://github.com/cpphoo/STARTUP.
arXiv commentPublished as a conference paper at ICLR 2021(oral)
References in corpus (12)
- Distilling the Knowledge in a Neural Network
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Deep Visual Domain Adaptation: A Survey
- Conditional Adversarial Domain Adaptation
- Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
- SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
- Learning to Self-Train for Semi-Supervised Few-Shot Classification
- Cross Attention Network for Few-shot Classification
- Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
- Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
- Domain Adaptive Transfer Learning with Specialist Models