Combating noisy labels by agreement: A joint training method with co-regularization

Abstract

Deep Learning with noisy labels is a practically challenging problem in weakly supervised learning. The state-of-the-art approaches "Decoupling" and "Co-teaching+" claim that the "disagreement" strategy is crucial for alleviating the problem of learning with noisy labels. In this paper, we start from a different perspective and propose a robust learning paradigm called JoCoR, which aims to reduce the diversity of two networks during training. Specifically, we first use two networks to make predictions on the same mini-batch data and calculate a joint loss with Co-Regularization for each training example. Then we select small-loss examples to update the parameters of both two networks simultaneously. Trained by the joint loss, these two networks would be more and more similar due to the effect of Co-Regularization. Extensive experimental results on corrupted data from benchmark datasets including MNIST, CIFAR-10, CIFAR-100 and Clothing1M demonstrate that JoCoR is superior to many state-of-the-art approaches for learning with noisy labels.

Accepted by CVPR 2020; Code is available at: https://github.com/hongxin001/JoCoR. arXiv admin note: text overlap with arXiv:1901.04215 by other authors

References in corpus (7)

• Understanding deep learning requires rethinking generalization
• Learning to Reweight Examples for Robust Deep Learning
• Classification with Noisy Labels by Importance Reweighting
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
• Probabilistic End-to-end Noise Correction for Learning with Noisy Labels

Cited by in corpus (6)

• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• Weak-shot Fine-grained Classification via Similarity Transfer
• Boosting Co-teaching with Compression Regularization for Label Noise
• Adaptive Sample Selection for Robust Learning under Label Noise
• Noisy Annotation Refinement for Object Detection