Dataset Bias in Few-shot Image Recognition

Abstract

The goal of few-shot image recognition (FSIR) is to identify novel categories with a small number of annotated samples by exploiting transferable knowledge from training data (base categories). Most current studies assume that the transferable knowledge can be well used to identify novel categories. However, such transferable capability may be impacted by the dataset bias, and this problem has rarely been investigated before. Besides, most of few-shot learning methods are biased to different datasets, which is also an important issue that needs to be investigated deeply. In this paper, we first investigate the impact of transferable capabilities learned from base categories. Specifically, we use the relevance to measure relationships between base categories and novel categories. Distributions of base categories are depicted via the instance density and category diversity. The FSIR model learns better transferable knowledge from relevant training data. In the relevant data, dense instances or diverse categories can further enrich the learned knowledge. Experimental results on different sub-datasets of ImagNet demonstrate category relevance, instance density and category diversity can depict transferable bias from base categories. Second, we investigate performance differences on different datasets from dataset structures and different few-shot learning methods. Specifically, we introduce image complexity, intra-concept visual consistency, and inter-concept visual similarity to quantify characteristics of dataset structures. We use these quantitative characteristics and four few-shot learning methods to analyze performance differences on five different datasets. Based on the experimental analysis, some insightful observations are obtained from the perspective of both dataset structures and few-shot learning methods. We hope these observations are useful to guide future FSIR research.

References in corpus (17)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• How transferable are features in deep neural networks?
• Learning Transferable Features with Deep Adaptation Networks
• Theoretical Models of Learning to Learn
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Frustratingly Simple Few-Shot Object Detection
• Delta-encoder: an effective sample synthesis method for few-shot object recognition
• FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
• Learning to Self-Train for Semi-Supervised Few-Shot Classification
• Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples
• Hierarchically Structured Meta-learning
• Reflections on Shannon Information: In search of a natural information-entropy for images
• MetaPix: Few-Shot Video Retargeting
• Graph Optimal Transport for Cross-Domain Alignment
• XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning