Masking: A New Perspective of Noisy Supervision

Abstract

It is important to learn various types of classifiers given training data with noisy labels. Noisy labels, in the most popular noise model hitherto, are corrupted from ground-truth labels by an unknown noise transition matrix. Thus, by estimating this matrix, classifiers can escape from overfitting those noisy labels. However, such estimation is practically difficult, due to either the indirect nature of two-step approaches, or not big enough data to afford end-to-end approaches. In this paper, we propose a human-assisted approach called Masking that conveys human cognition of invalid class transitions and naturally speculates the structure of the noise transition matrix. To this end, we derive a structure-aware probabilistic model incorporating a structure prior, and solve the challenges from structure extraction and structure alignment. Thanks to Masking, we only estimate unmasked noise transition probabilities and the burden of estimation is tremendously reduced. We conduct extensive experiments on CIFAR-10 and CIFAR-100 with three noise structures as well as the industrial-level Clothing1M with agnostic noise structure, and the results show that Masking can improve the robustness of classifiers significantly.

arXiv commentNIPS 2018 camera-ready version

References in corpus (13)

• Improved Training of Wasserstein GANs
• Learning to Reweight Examples for Robust Deep Learning
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Training Convolutional Networks with Noisy Labels
• A Closer Look at Memorization in Deep Networks
• WebVision Database: Visual Learning and Understanding from Web Data
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Joint Optimization Framework for Learning with Noisy Labels
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Learning from Noisy Labels with Distillation
• Learning From Noisy Large-Scale Datasets With Minimal Supervision
• Iterative Learning with Open-set Noisy Labels
• On the Convergence of A Family of Robust Losses for Stochastic Gradient Descent

Cited by in corpus (37)

• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Are Anchor Points Really Indispensable in Label-Noise Learning?
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Provably Consistent Partial-Label Learning
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• Unsupervised Domain Adaptation of Black-Box Source Models
• Training Binary Neural Networks through Learning with Noisy Supervision
• L_DMI: An Information-theoretic Noise-robust Loss Function
• Learning with Bounded Instance- and Label-dependent Label Noise
• Provably End-to-end Label-Noise Learning without Anchor Points
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Learning with Multiple Complementary Labels
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• Rethinking Importance Weighting for Deep Learning under Distribution Shift
• Contrastive Learning Improves Model Robustness Under Label Noise
• Learning from a Complementary-label Source Domain: Theory and Algorithms
• Learning Frequency Domain Approximation for Binary Neural Networks
• Federated Noisy Client Learning
• Derivative Manipulation for General Example Weighting
• Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation
• Collaborative Label Correction via Entropy Thresholding
• The Resistance to Label Noise in K-NN and DNN Depends on its Concentration
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• Safeguarded Dynamic Label Regression for Generalized Noisy Supervision
• Self-Supervised Person Detection in 2D Range Data using a Calibrated Camera
• Multiplicative Reweighting for Robust Neural Network Optimization
• Towards Robust Pattern Recognition: A Review
• ARM: A Confidence-Based Adversarial Reweighting Module for Coarse Semantic Segmentation
• Meta Label Correction for Noisy Label Learning
• Learning with Noisy Labels for Sentence-level Sentiment Classification
• Mitigating Memorization in Sample Selection for Learning with Noisy Labels
• Do We Really Need Gold Samples for Sample Weighting Under Label Noise?
• Truth Discovery in Sequence Labels from Crowds