DivideMix: Learning with Noisy Labels as Semi-supervised Learning

Abstract

Deep neural networks are known to be annotation-hungry. Numerous efforts have been devoted to reducing the annotation cost when learning with deep networks. Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data. In this work, we propose DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques. In particular, DivideMix models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner. To avoid confirmation bias, we simultaneously train two diverged networks where each network uses the dataset division from the other network. During the semi-supervised training phase, we improve the MixMatch strategy by performing label co-refinement and label co-guessing on labeled and unlabeled samples, respectively. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods. Code is available at https://github.com/LiJunnan1992/DivideMix .

References in corpus (10)

• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• Regularizing Neural Networks by Penalizing Confident Output Distributions
• WebVision Database: Visual Learning and Understanding from Web Data
• Unsupervised Label Noise Modeling and Loss Correction
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• Probabilistic End-to-end Noise Correction for Learning with Noisy Labels
• Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion
• Robust Learning from Untrusted Sources

Cited by in corpus (88)

• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• SPICE: Semantic Pseudo-labeling for Image Clustering
• LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
• Active label cleaning for improved dataset quality under resource constraints
• A Survey of Label-noise Representation Learning: Past, Present and Future
• Hybrid Dynamic Contrast and Probability Distillation for Unsupervised Person Re-Id
• Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Co-Correcting: Noise-tolerant Medical Image Classification via mutual Label Correction
• Understanding and Improving Early Stopping for Learning with Noisy Labels
• Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
• On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification
• Identifying Mislabeled Data using the Area Under the Margin Ranking
• Learning from Pixel-Level Label Noise: A New Perspective for Semi-Supervised Semantic Segmentation
• FINE Samples for Learning with Noisy Labels
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• Provably End-to-end Label-Noise Learning without Anchor Points
• The Sound Demixing Challenge 2023 $\unicode{x2013}$ Music Demixing Track
• Local Label Point Correction for Edge Detection of Overlapping Cervical Cells
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• Robust Long-Tailed Learning under Label Noise
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptation
• MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels
• One-Step Abductive Multi-Target Learning with Diverse Noisy Samples and Its Application to Tumour Segmentation for Breast Cancer
• Dirichlet-Based Prediction Calibration for Learning with Noisy Labels
• Variational Rectification Inference for Learning with Noisy Labels
• Handling Noisy Labels via One-Step Abductive Multi-Target Learning and Its Application to Helicobacter Pylori Segmentation
• Beyond Cats and Dogs: Semi-supervised Classification of fuzzy labels with overclustering
• Learning From Long-Tailed Data With Noisy Labels
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise
• Fuzzy Overclustering: Semi-Supervised Classification of Fuzzy Labels with Overclustering and Inverse Cross-Entropy
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• FedDebug: Systematic Debugging for Federated Learning Applications
• Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition
• Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition
• Class-Specific Distribution Alignment for Semi-Supervised Medical Image Classification
• SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation
• Weak-shot Fine-grained Classification via Similarity Transfer
• Improving Unsupervised Image Clustering With Robust Learning
• Learning Fast Sample Re-weighting Without Reward Data
• A Study on the Impact of Data Augmentation for Training Convolutional Neural Networks in the Presence of Noisy Labels
• Enhancing Sample Utilization in Noise-Robust Deep Metric Learning With Subgroup-Based Positive-Pair Selection
• Instance Correction for Learning with Open-set Noisy Labels
• Combining Self-Supervised and Supervised Learning with Noisy Labels
• Boosting Co-teaching with Compression Regularization for Label Noise
• Faster Meta Update Strategy for Noise-Robust Deep Learning
• Confidence Adaptive Regularization for Deep Learning with Noisy Labels
• Adaptive Sample Selection for Robust Learning under Label Noise
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels
• ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentation
• Approximating Instance-Dependent Noise via Instance-Confidence Embedding
• An Ensemble Noise-Robust K-fold Cross-Validation Selection Method for Noisy Labels
• Reduced-Lead ECG Classifier Model Trained with DivideMix and Model Ensemble
• NGC: A Unified Framework for Learning with Open-World Noisy Data
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• Semi-Supervised Crowd Counting from Unlabeled Data
• Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond
• Co-matching: Combating Noisy Labels by Augmentation Anchoring
• Hand-Object Contact Prediction via Motion-Based Pseudo-Labeling and Guided Progressive Label Correction
• Consistency Regularization Can Improve Robustness to Label Noise
• Noise-Resistant Deep Metric Learning with Probabilistic Instance Filtering
• Evaluating Self and Semi-Supervised Methods for Remote Sensing Segmentation Tasks
• GMM Discriminant Analysis with Noisy Label for Each Class
• Pseudo-Labeling Enhanced by Privileged Information and Its Application to In Situ Sequencing Images
• CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
• Combating Label Noise With A General Surrogate Model For Sample Selection
• Mixup Without Hesitation
• Noisy Label Classification using Label Noise Selection with Test-Time Augmentation Cross-Entropy and NoiseMix Learning
• Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awareness
• Transform consistency for learning with noisy labels
• The Victim and The Beneficiary: Exploiting a Poisoned Model to Train a Clean Model on Poisoned Data
• Constrained Instance and Class Reweighting for Robust Learning under Label Noise
• HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance
• Training Classifiers that are Universally Robust to All Label Noise Levels
• Multiplicative Reweighting for Robust Neural Network Optimization
• Robust Temporal Ensembling for Learning with Noisy Labels
• Learning advisor networks for noisy image classification
• Life is not black and white -- Combining Semi-Supervised Learning with fuzzy labels
• INN: A Method Identifying Clean-annotated Samples via Consistency Effect in Deep Neural Networks
• Sum of Ranked Range Loss for Supervised Learning
• Friends and Foes in Learning from Noisy Labels
• Probabilistic Decoupling of Labels in Classification