An empirical study of pretrained representations for few-shot classification
arXiv:1910.01319
Abstract
Recent algorithms with state-of-the-art few-shot classification results start their procedure by computing data features output by a large pretrained model. In this paper we systematically investigate which models provide the best representations for a few-shot image classification task when pretrained on the Imagenet dataset. We test their representations when used as the starting point for different few-shot classification algorithms. We observe that models trained on a supervised classification task have higher performance than models trained in an unsupervised manner even when transferred to out-of-distribution datasets. Models trained with adversarial robustness transfer better, while having slightly lower accuracy than supervised models.
References in corpus (18)
- Scikit-learn: Machine Learning in Python
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Fine-Grained Visual Classification of Aircraft
- Matching Networks for One Shot Learning
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Meta-Learning for Semi-Supervised Few-Shot Classification
- Learning Representations by Maximizing Mutual Information Across Views
- Do ImageNet Classifiers Generalize to ImageNet?
- Adversarial Examples Are Not Bugs, They Are Features
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
- Meta-Learning with Latent Embedding Optimization
- Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
- Feature Denoising for Improving Adversarial Robustness
- Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples
- Adversarial Robustness as a Prior for Learned Representations
- Conditional Neural Processes
- Adaptive Posterior Learning: few-shot learning with a surprise-based memory module