Meta Transition Adaptation for Robust Deep Learning with Noisy Labels

Abstract

To discover intrinsic inter-class transition probabilities underlying data, learning with noise transition has become an important approach for robust deep learning on corrupted labels. Prior methods attempt to achieve such transition knowledge by pre-assuming strongly confident anchor points with 1-probability belonging to a specific class, generally infeasible in practice, or directly jointly estimating the transition matrix and learning the classifier from the noisy samples, always leading to inaccurate estimation misguided by wrong annotation information especially in large noise cases. To alleviate these issues, this study proposes a new meta-transition-learning strategy for the task. Specifically, through the sound guidance of a small set of meta data with clean labels, the noise transition matrix and the classifier parameters can be mutually ameliorated to avoid being trapped by noisy training samples, and without need of any anchor point assumptions. Besides, we prove our method is with statistical consistency guarantee on correctly estimating the desired transition matrix. Extensive synthetic and real experiments validate that our method can more accurately extract the transition matrix, naturally following its more robust performance than prior arts. Its essential relationship with label distribution learning is also discussed, which explains its fine performance even under no-noise scenarios.

14 pages

References in corpus (9)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Distilling the Knowledge in a Neural Network
• mixup: Beyond Empirical Risk Minimization
• Theoretical Models of Learning to Learn
• Unsupervised Label Noise Modeling and Loss Correction
• How does Disagreement Help Generalization against Label Corruption?
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• Small Sample Learning in Big Data Era

Cited by in corpus (7)

• Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation
• Instance Correction for Learning with Open-set Noisy Labels
• Faster Meta Update Strategy for Noise-Robust Deep Learning
• Contrastive Representations for Label Noise Require Fine-Tuning
• Alleviating Noisy-label Effects in Image Classification via Probability Transition Matrix