Robust Inference via Generative Classifiers for Handling Noisy Labels

Abstract

Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets. To mitigate the issue, we propose a novel inference method, termed Robust Generative classifier (RoG), applicable to any discriminative (e.g., softmax) neural classifier pre-trained on noisy datasets. In particular, we induce a generative classifier on top of hidden feature spaces of the pre-trained DNNs, for obtaining a more robust decision boundary. By estimating the parameters of generative classifier using the minimum covariance determinant estimator, we significantly improve the classification accuracy with neither re-training of the deep model nor changing its architectures. With the assumption of Gaussian distribution for features, we prove that RoG generalizes better than baselines under noisy labels. Finally, we propose the ensemble version of RoG to improve its performance by investigating the layer-wise characteristics of DNNs. Our extensive experimental results demonstrate the superiority of RoG given different learning models optimized by several training techniques to handle diverse scenarios of noisy labels.

Accepted in ICML 2019

Cited by in corpus (19)

• How does Disagreement Help Generalization against Label Corruption?
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
• Learning from Failure: Training Debiased Classifier from Biased Classifier
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• FINE Samples for Learning with Noisy Labels
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• Self-Contrastive Forward-Forward Algorithm
• Learning Fast Sample Re-weighting Without Reward Data
• Derivative Manipulation for General Example Weighting
• Adversarial Partial Multi-Label Learning
• Countering Noisy Labels By Learning From Auxiliary Clean Labels
• NGC: A Unified Framework for Learning with Open-World Noisy Data
• Deep k-NN for Noisy Labels
• Multi-Label Gold Asymmetric Loss Correction with Single-Label Regulators
• Robust Temporal Ensembling for Learning with Noisy Labels
• Learning Task-aware Robust Deep Learning Systems
• Using Under-trained Deep Ensembles to Learn Under Extreme Label Noise
• One-shot Weakly-Supervised Segmentation in Medical Images