Are Anchor Points Really Indispensable in Label-Noise Learning?

Abstract

In label-noise learning, \textit{noise transition matrix}, denoting the probabilities that clean labels flip into noisy labels, plays a central role in building \textit{statistically consistent classifiers}. Existing theories have shown that the transition matrix can be learned by exploiting \textit{anchor points} (i.e., data points that belong to a specific class almost surely). However, when there are no anchor points, the transition matrix will be poorly learned, and those current consistent classifiers will significantly degenerate. In this paper, without employing anchor points, we propose a \textit{transition-revision} ($T$-Revision) method to effectively learn transition matrices, leading to better classifiers. Specifically, to learn a transition matrix, we first initialize it by exploiting data points that are similar to anchor points, having high \textit{noisy class posterior probabilities}. Then, we modify the initialized matrix by adding a \textit{slack variable}, which can be learned and validated together with the classifier by using noisy data. Empirical results on benchmark-simulated and real-world label-noise datasets demonstrate that without using exact anchor points, the proposed method is superior to the state-of-the-art label-noise learning methods.

Accepted by NeurIPS 2019

References in corpus (9)

• Learning to Reweight Examples for Robust Deep Learning
• Classification with Noisy Labels by Importance Reweighting
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
• Dimensionality-Driven Learning with Noisy Labels
• Masking: A New Perspective of Noisy Supervision
• Mixture Proportion Estimation via Kernel Embedding of Distributions
• Robustness of Conditional GANs to Noisy Labels
• Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels

Cited by in corpus (30)

• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Provably Consistent Partial-Label Learning
• Understanding and Improving Early Stopping for Learning with Noisy Labels
• Progressive Identification of True Labels for Partial-Label Learning
• Instance-dependent Label-noise Learning under a Structural Causal Model
• Trustable Co-label Learning from Multiple Noisy Annotators
• Provably End-to-end Label-Noise Learning without Anchor Points
• Learning with Instance-Dependent Label Noise: A Sample Sieve Approach
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Learning with Multiple Complementary Labels
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels
• MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels
• Rethinking Importance Weighting for Deep Learning under Distribution Shift
• Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification
• Faster Meta Update Strategy for Noise-Robust Deep Learning
• Confidence Adaptive Regularization for Deep Learning with Noisy Labels
• Complementary to Multiple Labels: A Correlation-Aware Correction Approach
• Multi-Class Classification from Noisy-Similarity-Labeled Data
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• No Regret Sample Selection with Noisy Labels
• Pointwise Binary Classification with Pairwise Confidence Comparisons
• Meta Label Correction for Noisy Label Learning
• On the Robustness of Average Losses for Partial-Label Learning
• CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noise
• Adjustable Robust Transformer for High Myopia Screening in Optical Coherence Tomography