Combining Self-Supervised and Supervised Learning with Noisy Labels
arXiv:2011.08145
Abstract
Since convolutional neural networks (CNNs) can easily overfit noisy labels, which are ubiquitous in visual classification tasks, it has been a great challenge to train CNNs against them robustly. Various methods have been proposed for this challenge. However, none of them pay attention to the difference between representation and classifier learning of CNNs. Thus, inspired by the observation that classifier is more robust to noisy labels while representation is much more fragile, and by the recent advances of self-supervised representation learning (SSRL) technologies, we design a new method, i.e., CSNL, to obtain representation by SSRL without labels and train the classifier directly with noisy labels. Extensive experiments are performed on both synthetic and real benchmark datasets. Results demonstrate that the proposed method can beat the state-of-the-art ones by a large margin, especially under a high noisy level.
References in corpus (11)
- A Simple Framework for Contrastive Learning of Visual Representations
- Bootstrap your own latent: A new approach to self-supervised Learning
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- DivideMix: Learning with Noisy Labels as Semi-supervised Learning
- Rethinking Pre-training and Self-training
- How does Disagreement Help Generalization against Label Corruption?
- Robust Loss Functions under Label Noise for Deep Neural Networks
- A Survey of Label-noise Representation Learning: Past, Present and Future
- Contrastive Adaptation Network for Unsupervised Domain Adaptation
- Probabilistic End-to-end Noise Correction for Learning with Noisy Labels
- Neural Graph Machines: Learning Neural Networks Using Graphs