Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation
arXiv:2111.10932
Abstract
As the adoption of deep learning techniques in industrial applications grows with increasing speed and scale, successful deployment of deep learning models often hinges on the availability, volume, and quality of annotated data. In this paper, we tackle the problems of efficient data labeling and annotation verification under the human-in-the-loop setting. We showcase that the latest advancements in the field of self-supervised visual representation learning can lead to tools and methods that benefit the curation and engineering of natural image datasets, reducing annotation cost and increasing annotation quality. We propose a unifying framework by leveraging self-supervised semi-supervised learning and use it to construct workflows for data labeling and annotation verification tasks. We demonstrate the effectiveness of our workflows over existing methodologies. On active learning task, our method achieves 97.0% Top-1 Accuracy on CIFAR10 with 0.1% annotated data, and 83.9% Top-1 Accuracy on CIFAR100 with 10% annotated data. When learning with 50% of wrong labels, our method achieves 97.4% Top-1 Accuracy on CIFAR10 and 85.5% Top-1 Accuracy on CIFAR100.
arXiv commentAccepted to NeurIPS 2021 DCAI Workshop
References in corpus (16)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Decoupled Weight Decay Regularization
- Learning Transferable Visual Models From Natural Language Supervision
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Bootstrap your own latent: A new approach to self-supervised Learning
- Improved Baselines with Momentum Contrastive Learning
- Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Semi-Supervised Learning with Deep Generative Models
- BEiT: BERT Pre-Training of Image Transformers
- Interpolation Consistency Training for Semi-Supervised Learning
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Semi-Supervised Learning with Ladder Networks
- Semi-Supervised Learning with Generative Adversarial Networks
- There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
- Deep Metric Transfer for Label Propagation with Limited Annotated Data