Learning with Feature-Dependent Label Noise: A Progressive Approach

Abstract

Label noise is frequently observed in real-world large-scale datasets. The noise is introduced due to a variety of reasons; it is heterogeneous and feature-dependent. Most existing approaches to handling noisy labels fall into two categories: they either assume an ideal feature-independent noise, or remain heuristic without theoretical guarantees. In this paper, we propose to target a new family of feature-dependent label noise, which is much more general than commonly used i.i.d. label noise and encompasses a broad spectrum of noise patterns. Focusing on this general noise family, we propose a progressive label correction algorithm that iteratively corrects labels and refines the model. We provide theoretical guarantees showing that for a wide variety of (unknown) noise patterns, a classifier trained with this strategy converges to be consistent with the Bayes classifier. In experiments, our method outperforms SOTA baselines and is robust to various noise types and levels.

ICLR 2021 (Spotlight)

References in corpus (6)

• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Noise Tolerance under Risk Minimization
• Making Risk Minimization Tolerant to Label Noise
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• Combating Label Noise in Deep Learning Using Abstention
• On the Resistance of Nearest Neighbor to Random Noisy Labels

Cited by in corpus (11)

• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
• Boosting Co-teaching with Compression Regularization for Label Noise
• Instance Correction for Learning with Open-set Noisy Labels
• Hand-Object Contact Prediction via Motion-Based Pseudo-Labeling and Guided Progressive Label Correction
• Robust Partial-Label Learning by Leveraging Class Activation Values
• Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awareness
• Learning to Rectify for Robust Learning with Noisy Labels
• CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
• One-shot Weakly-Supervised Segmentation in Medical Images