When Does Self-supervision Improve Few-shot Learning?

Abstract

We investigate the role of self-supervised learning (SSL) in the context of few-shot learning. Although recent research has shown the benefits of SSL on large unlabeled datasets, its utility on small datasets is relatively unexplored. We find that SSL reduces the relative error rate of few-shot meta-learners by 4%-27%, even when the datasets are small and only utilizing images within the datasets. The improvements are greater when the training set is smaller or the task is more challenging. Although the benefits of SSL may increase with larger training sets, we observe that SSL can hurt the performance when the distributions of images used for meta-learning and SSL are different. We conduct a systematic study by varying the degree of domain shift and analyzing the performance of several meta-learners on a multitude of domains. Based on this analysis we present a technique that automatically selects images for SSL from a large, generic pool of unlabeled images for a given dataset that provides further improvements.

ECCV 2020 camera ready. This is an updated version of "Boosting Supervision with Self-Supervision for Few-shot Learning" arXiv:1906.07079

References in corpus (10)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning deep representations by mutual information estimation and maximization
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Learning Representations by Maximizing Mutual Information Across Views
• DropBlock: A regularization method for convolutional networks
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• Multi-Task Learning as Multi-Objective Optimization
• A critical analysis of self-supervision, or what we can learn from a single image
• Domain Adaptive Transfer Learning with Specialist Models
• Selfie: Self-supervised Pretraining for Image Embedding

Cited by in corpus (7)

• When Does Self-Supervision Help Graph Convolutional Networks?
• Learning from Few Samples: A Survey
• Improving out-of-distribution generalization via multi-task self-supervised pretraining
• To Balance or Not to Balance: A Simple-yet-Effective Approach for Learning with Long-Tailed Distributions
• Few-Shot Electronic Health Record Coding through Graph Contrastive Learning
• Multi-Pretext Attention Network for Few-shot Learning with Self-supervision
• Reinforced Attention for Few-Shot Learning and Beyond