Robust Learning Under Label Noise With Iterative Noise-Filtering
arXiv:1906.00216
Abstract
We consider the problem of training a model under the presence of label noise. Current approaches identify samples with potentially incorrect labels and reduce their influence on the learning process by either assigning lower weights to them or completely removing them from the training set. In the first case the model however still learns from noisy labels; in the latter approach, good training data can be lost. In this paper, we propose an iterative semi-supervised mechanism for robust learning which excludes noisy labels but is still able to learn from the corresponding samples. To this end, we add an unsupervised loss term that also serves as a regularizer against the remaining label noise. We evaluate our approach on common classification tasks with different noise ratios. Our robust models outperform the state-of-the-art methods by a large margin. Especially for very large noise ratios, we achieve up to 20 % absolute improvement compared to the previous best model.
References in corpus (13)
- Distilling the Knowledge in a Neural Network
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Semi-Supervised Learning with Deep Generative Models
- Learning to Reweight Examples for Robust Deep Learning
- Semi-Supervised Learning with Ladder Networks
- Training Deep Neural Networks on Noisy Labels with Bootstrapping
- MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
- Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
- Shake-Shake regularization
- Iterative Learning with Open-set Noisy Labels
- Auxiliary Image Regularization for Deep CNNs with Noisy Labels