Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels

Abstract

Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) as DNNs usually have the high capacity to memorize the noisy labels. In this paper, we find that the test accuracy can be quantitatively characterized in terms of the noise ratio in datasets. In particular, the test accuracy is a quadratic function of the noise ratio in the case of symmetric noise, which explains the experimental findings previously published. Based on our analysis, we apply cross-validation to randomly split noisy datasets, which identifies most samples that have correct labels. Then we adopt the Co-teaching strategy which takes full advantage of the identified samples to train DNNs robustly against noisy labels. Compared with extensive state-of-the-art methods, our strategy consistently improves the generalization performance of DNNs under both synthetic and real-world training noise.

Correspondence to: Guangyong Chen <[email protected]>

Cited by in corpus (22)

• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• A Survey of Label-noise Representation Learning: Past, Present and Future
• MoPro: Webly Supervised Learning with Momentum Prototypes
• Towards Noise-resistant Object Detection with Noisy Annotations
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise
• Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterion
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Learning Fast Sample Re-weighting Without Reward Data
• Coresets for Robust Training of Neural Networks against Noisy Labels
• Instance Correction for Learning with Open-set Noisy Labels
• Confidence Adaptive Regularization for Deep Learning with Noisy Labels
• Which Strategies Matter for Noisy Label Classification? Insight into Loss and Uncertainty
• A Meta Approach to Defend Noisy Labels by the Manifold Regularizer PSDR
• NGC: A Unified Framework for Learning with Open-World Noisy Data
• Co-matching: Combating Noisy Labels by Augmentation Anchoring
• Multi-Label Gold Asymmetric Loss Correction with Single-Label Regulators
• P-DIFF: Learning Classifier with Noisy Labels based on Probability Difference Distributions
• Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awareness
• INN: A Method Identifying Clean-annotated Samples via Consistency Effect in Deep Neural Networks
• In-Loop Meta-Learning with Gradient-Alignment Reward
• Learning to Selectively Learn for Weakly-supervised Paraphrase Generation
• Mitigating Memorization in Sample Selection for Learning with Noisy Labels