Reliable Label Bootstrapping for Semi-Supervised Learning

Abstract

Reducing the amount of labels required to train convolutional neural networks without performance degradation is key to effectively reduce human annotation efforts. We propose Reliable Label Bootstrapping (ReLaB), an unsupervised preprossessing algorithm which improves the performance of semi-supervised algorithms in extremely low supervision settings. Given a dataset with few labeled samples, we first learn meaningful self-supervised, latent features for the data. Second, a label propagation algorithm propagates the known labels on the unsupervised features, effectively labeling the full dataset in an automatic fashion. Third, we select a subset of correctly labeled (reliable) samples using a label noise detection algorithm. Finally, we train a semi-supervised algorithm on the extended subset. We show that the selection of the network architecture and the self-supervised algorithm are important factors to achieve successful label propagation and demonstrate that ReLaB substantially improves semi-supervised learning in scenarios of very limited supervision on CIFAR-10, CIFAR-100 and mini-ImageNet. We reach average error rates of $22.34$ with 1 random labeled sample per class on CIFAR-10 and lower this error to $8.46$ when the labeled sample in each class is highly representative. Our work is fully reproducible: https://github.com/PaulAlbert31/ReLaB.

10 pages, 3 figures

References in corpus (25)

• Deep Residual Learning for Image Recognition
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• Unsupervised Data Augmentation for Consistency Training
• Unsupervised Representation Learning by Predicting Image Rotations
• Matching Networks for One Shot Learning
• AutoAugment: Learning Augmentation Policies from Data
• Semi-Supervised Learning with Ladder Networks
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Deep Clustering for Unsupervised Learning of Visual Features
• Vision Meets Drones: A Challenge
• WebVision Database: Visual Learning and Understanding from Web Data
• Unsupervised Visual Representation Learning by Context Prediction
• Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
• Aggregating Deep Convolutional Features for Image Retrieval
• Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Joint Optimization Framework for Learning with Noisy Labels
• Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
• Revisiting Self-Supervised Visual Representation Learning
• Unsupervised Deep Learning by Neighbourhood Discovery
• Probabilistic End-to-end Noise Correction for Learning with Noisy Labels
• Learning by Association - A versatile semi-supervised training method for neural networks
• AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data
• Iterative Learning with Open-set Noisy Labels
• A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels