Out-distribution aware Self-training in an Open World Setting
arXiv:2012.12372
Abstract
Deep Learning heavily depends on large labeled datasets which limits further improvements. While unlabeled data is available in large amounts, in particular in image recognition, it does not fulfill the closed world assumption of semi-supervised learning that all unlabeled data are task-related. The goal of this paper is to leverage unlabeled data in an open world setting to further improve prediction performance. For this purpose, we introduce out-distribution aware self-training, which includes a careful sample selection strategy based on the confidence of the classifier. While normal self-training deteriorates prediction performance, our iterative scheme improves using up to 15 times the amount of originally labeled data. Moreover, our classifiers are by design out-distribution aware and can thus distinguish task-related inputs from unrelated ones.
References in corpus (13)
- Improved Regularization of Convolutional Neural Networks with Cutout
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Deep Anomaly Detection with Outlier Exposure
- Billion-scale semi-supervised learning for image classification
- ShakeDrop Regularization for Deep Residual Learning
- There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
- Do CIFAR-10 Classifiers Generalize to CIFAR-10?
- FMix: Enhancing Mixed Sample Data Augmentation
- Outlier Exposure with Confidence Control for Out-of-Distribution Detection
- Large image datasets: A pyrrhic win for computer vision?
- EDF: Ensemble, Distill, and Fuse for Easy Video Labeling