Learning with Instance-Dependent Label Noise: A Sample Sieve Approach

Abstract

Human-annotated labels are often prone to noise, and the presence of such noise will degrade the performance of the resulting deep neural network (DNN) models. Much of the literature (with several recent exceptions) of learning with noisy labels focuses on the case when the label noise is independent of features. Practically, annotations errors tend to be instance-dependent and often depend on the difficulty levels of recognizing a certain task. Applying existing results from instance-independent settings would require a significant amount of estimation of noise rates. Therefore, providing theoretically rigorous solutions for learning with instance-dependent label noise remains a challenge. In this paper, we propose CORES${}^2$ (COnfidence REgularized Sample Sieve), which progressively sieves out corrupted examples. The implementation of CORES${}^2$ does not require specifying noise rates and yet we are able to provide theoretical guarantees of CORES${}^2$ in filtering out the corrupted examples. This high-quality sample sieve allows us to treat clean examples and the corrupted ones separately in training a DNN solution, and such a separation is shown to be advantageous in the instance-dependent noise setting. We demonstrate the performance of CORES${}^2$ on CIFAR10 and CIFAR100 datasets with synthetic instance-dependent label noise and Clothing1M with real-world human noise. As of independent interests, our sample sieve provides a generic machinery for anatomizing noisy datasets and provides a flexible interface for various robust training techniques to further improve the performance. Code is available at https://github.com/UCSC-REAL/cores.

arXiv commentICLR 2021

References in corpus (15)

• Unsupervised Data Augmentation for Consistency Training
• Understanding deep learning requires rethinking generalization
• Classification with Noisy Labels by Importance Reweighting
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Noise Tolerance under Risk Minimization
• Are Anchor Points Really Indispensable in Label-Noise Learning?
• How does Disagreement Help Generalization against Label Corruption?
• SELF: Learning to Filter Noisy Labels with Self-Ensembling
• Confidence Scores Make Instance-dependent Label-noise Learning Possible
• Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels
• Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization
• Learning Adaptive Loss for Robust Learning with Noisy Labels
• Rethinking Importance Weighting for Deep Learning under Distribution Shift
• Weak and Strong Gradient Directions: Explaining Memorization, Generalization, and Hardness of Examples at Scale

Cited by in corpus (4)

• The Sound Demixing Challenge 2023 $\unicode{x2013}$ Music Demixing Track
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• Joint Noise-Tolerant Learning and Meta Camera Shift Adaptation for Unsupervised Person Re-Identification
• Variational Rectification Inference for Learning with Noisy Labels