Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection

Abstract

Recent studies on learning with noisy labels have shown remarkable performance by exploiting a small clean dataset. In particular, model agnostic meta-learning-based label correction methods further improve performance by correcting noisy labels on the fly. However, there is no safeguard on the label miscorrection, resulting in unavoidable performance degradation. Moreover, every training step requires at least three back-propagations, significantly slowing down the training speed. To mitigate these issues, we propose a robust and efficient method that learns a label transition matrix on the fly. Employing the transition matrix makes the classifier skeptical about all the corrected samples, which alleviates the miscorrection issue. We also introduce a two-head architecture to efficiently estimate the label transition matrix every iteration within a single back-propagation, so that the estimated matrix closely follows the shifting noise distribution induced by label correction. Extensive experiments demonstrate that our approach shows the best performance in training efficiency while having comparable or better accuracy than existing methods.

arXiv commentECCV2022

References in corpus (27)

• mixup: Beyond Empirical Risk Minimization
• On Calibration of Modern Neural Networks
• Learning to Reweight Examples for Robust Deep Learning
• Classification with Noisy Labels by Importance Reweighting
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Training Convolutional Networks with Noisy Labels
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• A Closer Look at Memorization in Deep Networks
• A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
• Dimensionality-Driven Learning with Noisy Labels
• Are Anchor Points Really Indispensable in Label-Noise Learning?
• Masking: A New Perspective of Noisy Supervision
• Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• Normalized Loss Functions for Deep Learning with Noisy Labels
• Combating Label Noise in Deep Learning Using Abstention
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• Robustness of Conditional GANs to Noisy Labels
• SIGUA: Forgetting May Make Learning with Noisy Labels More Robust
• Identifying Mislabeled Data using the Area Under the Margin Ranking
• Training Binary Neural Networks through Learning with Noisy Supervision
• How does Early Stopping Help Generalization against Label Noise?
• A Topological Filter for Learning with Label Noise
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Augmentation Strategies for Learning with Noisy Labels
• Learning to Purify Noisy Labels via Meta Soft Label Corrector