Free Lunch for Few-shot Learning: Distribution Calibration

Abstract

Learning from a limited number of samples is challenging since the learned model can easily become overfitted based on the biased distribution formed by only a few training examples. In this paper, we calibrate the distribution of these few-sample classes by transferring statistics from the classes with sufficient examples, then an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. We assume every dimension in the feature representation follows a Gaussian distribution so that the mean and the variance of the distribution can borrow from that of similar classes whose statistics are better estimated with an adequate number of samples. Our method can be built on top of off-the-shelf pretrained feature extractors and classification models without extra parameters. We show that a simple logistic regression classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy on two datasets (~5% improvement on miniImageNet compared to the next best). The visualization of these generated features demonstrates that our calibrated distribution is an accurate estimation.

arXiv commentICLR 2021

References in corpus (12)

• Scikit-learn: Machine Learning in Python
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Delta-encoder: an effective sample synthesis method for few-shot object recognition
• An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning
• Low-Shot Learning from Imaginary Data
• Implicit Semantic Data Augmentation for Deep Networks
• Learning to Propagate for Graph Meta-Learning
• Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective
• Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentation
• Diversity Helps: Unsupervised Few-shot Learning via Distribution Shift-based Data Augmentation

Cited by in corpus (14)

• Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
• GCT: Graph Co-Training for Semi-Supervised Few-Shot Learning
• Objects in Semantic Topology
• LibFewShot: A Comprehensive Library for Few-shot Learning
• A Closer Look at Prototype Classifier for Few-shot Image Classification
• ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning
• Instance Correction for Learning with Open-set Noisy Labels
• Label-Aware Distribution Calibration for Long-tailed Classification
• On the Importance of Firth Bias Reduction in Few-Shot Classification
• TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation
• Improving Forward Compatibility in Class Incremental Learning by Increasing Representation Rank and Feature Richness
• Few-shot Learning with Global Relatedness Decoupled-Distillation
• MDFM: Multi-Decision Fusing Model for Few-Shot Learning
• Towards Enabling Meta-Learning from Target Models