DivideMix: Learning with Noisy Labels as Semi-supervised Learning

Abstract

Deep neural networks are known to be annotation-hungry. Numerous efforts have been devoted to reducing the annotation cost when learning with deep networks. Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data. In this work, we propose DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques. In particular, DivideMix models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner. To avoid confirmation bias, we simultaneously train two diverged networks where each network uses the dataset division from the other network. During the semi-supervised training phase, we improve the MixMatch strategy by performing label co-refinement and label co-guessing on labeled and unlabeled samples, respectively. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods. Code is available at https://github.com/LiJunnan1992/DivideMix .

References in corpus (7)

• Regularizing Neural Networks by Penalizing Confident Output Distributions
• WebVision Database: Visual Learning and Understanding from Web Data
• Unsupervised Label Noise Modeling and Loss Correction
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• Probabilistic End-to-end Noise Correction for Learning with Noisy Labels
• Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion
• Robust Learning from Untrusted Sources

Cited by in corpus (15)

• A Survey of Label-noise Representation Learning: Past, Present and Future
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise
• Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition
• SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Improving Unsupervised Image Clustering With Robust Learning
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels
• Approximating Instance-Dependent Noise via Instance-Confidence Embedding
• Co-matching: Combating Noisy Labels by Augmentation Anchoring
• Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond
• Mixup Without Hesitation
• Transform consistency for learning with noisy labels
• Probabilistic Decoupling of Labels in Classification
• Friends and Foes in Learning from Noisy Labels