CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning

Abstract

Labels are costly and sometimes unreliable. Noisy label learning, semi-supervised learning, and contrastive learning are three different strategies for designing learning processes requiring less annotation cost. Semi-supervised learning and contrastive learning have been recently demonstrated to improve learning strategies that address datasets with noisy labels. Still, the inner connections between these fields as well as the potential to combine their strengths together have only started to emerge. In this paper, we explore further ways and advantages to fuse them. Specifically, we propose CSSL, a unified Contrastive Semi-Supervised Learning algorithm, and CoDiM (Contrastive DivideMix), a novel algorithm for learning with noisy labels. CSSL leverages the power of classical semi-supervised learning and contrastive learning technologies and is further adapted to CoDiM, which learns robustly from multiple types and levels of label noise. We show that CoDiM brings consistent improvements and achieves state-of-the-art results on multiple benchmarks.

19 Pages, 9 figures, conference paper

References in corpus (15)

• Bootstrap your own latent: A new approach to self-supervised Learning
• Understanding deep learning requires rethinking generalization
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Exploring Simple Siamese Representation Learning
• WebVision Database: Visual Learning and Understanding from Web Data
• Unsupervised Label Noise Modeling and Loss Correction
• How does Disagreement Help Generalization against Label Corruption?
• LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
• Learning with Feature-Dependent Label Noise: A Progressive Approach
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• Combining Self-Supervised and Supervised Learning with Noisy Labels