A Broader Study of Cross-Domain Few-Shot Learning

Abstract

Recent progress on few-shot learning largely relies on annotated data for meta-learning: base classes sampled from the same domain as the novel classes. However, in many applications, collecting data for meta-learning is infeasible or impossible. This leads to the cross-domain few-shot learning problem, where there is a large shift between base and novel class domains. While investigations of the cross-domain few-shot scenario exist, these works are limited to natural images that still contain a high degree of visual similarity. No work yet exists that examines few-shot learning across different imaging methods seen in real world scenarios, such as aerial and medical imaging. In this paper, we propose the Broader Study of Cross-Domain Few-Shot Learning (BSCD-FSL) benchmark, consisting of image data from a diverse assortment of image acquisition methods. This includes natural images, such as crop disease images, but additionally those that present with an increasing dissimilarity to natural images, such as satellite images, dermatology images, and radiology images. Extensive experiments on the proposed benchmark are performed to evaluate state-of-art meta-learning approaches, transfer learning approaches, and newer methods for cross-domain few-shot learning. The results demonstrate that state-of-art meta-learning methods are surprisingly outperformed by earlier meta-learning approaches, and all meta-learning methods underperform in relation to simple fine-tuning by 12.8% average accuracy. Performance gains previously observed with methods specialized for cross-domain few-shot learning vanish in this more challenging benchmark. Finally, accuracy of all methods tend to correlate with dataset similarity to natural images, verifying the value of the benchmark to better represent the diversity of data seen in practice and guiding future research.

arXiv commentECCV 2020. Website: https://www.learning-with-limited-labels.com/

References in corpus (17)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• How transferable are features in deep neural networks?
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• Meta-Learning for Semi-Supervised Few-Shot Classification
• Domain Separation Networks
• On First-Order Meta-Learning Algorithms
• Few-Shot Learning with Graph Neural Networks
• Delta-encoder: an effective sample synthesis method for few-shot object recognition
• Learning to Self-Train for Semi-Supervised Few-Shot Classification
• Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
• Co-regularized Alignment for Unsupervised Domain Adaptation
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Are All Layers Created Equal?
• Low-Shot Learning from Imaginary Data
• Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions
• Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasets

Cited by in corpus (9)

• Feature Transformation Ensemble Model with Batch Spectral Regularization for Cross-Domain Few-Shot Classification
• Cross-Domain Few-Shot Learning by Representation Fusion
• Explanation-Guided Training for Cross-Domain Few-Shot Classification
• Cross-Domain Few-Shot Learning with Meta Fine-Tuning
• Cross-domain few-shot learning with unlabelled data
• Large Margin Mechanism and Pseudo Query Set on Cross-Domain Few-Shot Learning
• Machine learning with limited data
• Ensemble Model with Batch Spectral Regularization and Data Blending for Cross-Domain Few-Shot Learning with Unlabeled Data
• Prediction of neonatal mortality in Sub-Saharan African countries using data-level linkage of multiple surveys