Learning to Self-Train for Semi-Supervised Few-Shot Classification

Abstract

Few-shot classification (FSC) is challenging due to the scarcity of labeled training data (e.g. only one labeled data point per class). Meta-learning has shown to achieve promising results by learning to initialize a classification model for FSC. In this paper we propose a novel semi-supervised meta-learning method called learning to self-train (LST) that leverages unlabeled data and specifically meta-learns how to cherry-pick and label such unsupervised data to further improve performance. To this end, we train the LST model through a large number of semi-supervised few-shot tasks. On each task, we train a few-shot model to predict pseudo labels for unlabeled data, and then iterate the self-training steps on labeled and pseudo-labeled data with each step followed by fine-tuning. We additionally learn a soft weighting network (SWN) to optimize the self-training weights of pseudo labels so that better ones can contribute more to gradient descent optimization. We evaluate our LST method on two ImageNet benchmarks for semi-supervised few-shot classification and achieve large improvements over the state-of-the-art method. Code is at https://github.com/xinzheli1217/learning-to-self-train.

33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

References in corpus (4)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Delta-encoder: an effective sample synthesis method for few-shot object recognition
• Low-Shot Learning from Imaginary Data

Cited by in corpus (32)

• Self-training with Noisy Student improves ImageNet classification
• Adaptive Aggregation Networks for Class-Incremental Learning
• An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning
• Transductive Information Maximization For Few-Shot Learning
• DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning
• Adaptive Self-training for Few-shot Neural Sequence Labeling
• Meta-Learned Confidence for Few-shot Learning
• Part-aware Prototype Network for Few-shot Semantic Segmentation
• Explanation-Guided Training for Cross-Domain Few-Shot Classification
• Self-training for Few-shot Transfer Across Extreme Task Differences
• Towards Cross-Granularity Few-Shot Learning: Coarse-to-Fine Pseudo-Labeling with Visual-Semantic Meta-Embedding
• A Broader Study of Cross-Domain Few-Shot Learning
• Prototype Rectification for Few-Shot Learning
• Semi-supervised Open-World Object Detection
• Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining
• Instance Credibility Inference for Few-Shot Learning
• Learning to Impute: A General Framework for Semi-supervised Learning
• Looking back to lower-level information in few-shot learning
• Semi-Supervised Few-Shot Learning with Prototypical Random Walks
• FOAL: Fast Online Adaptive Learning for Cardiac Motion Estimation
• ECACL: A Holistic Framework for Semi-Supervised Domain Adaptation
• Semi-Supervised Wide-Angle Portraits Correction by Multi-Scale Transformer
• What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data
• Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
• How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning
• Semi-supervised Relation Extraction via Incremental Meta Self-Training
• Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation
• Task-Adaptive Clustering for Semi-Supervised Few-Shot Classification
• Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks
• Learning Improved Representations by Transferring Incomplete Evidence Across Heterogeneous Tasks
• Dataset Bias in Few-shot Image Recognition
• LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training