Unsupervised Label Noise Modeling and Loss Correction

Abstract

Despite being robust to small amounts of label noise, convolutional neural networks trained with stochastic gradient methods have been shown to easily fit random labels. When there are a mixture of correct and mislabelled targets, networks tend to fit the former before the latter. This suggests using a suitable two-component mixture model as an unsupervised generative model of sample loss values during training to allow online estimation of the probability that a sample is mislabelled. Specifically, we propose a beta mixture to estimate this probability and correct the loss by relying on the network prediction (the so-called bootstrapping loss). We further adapt mixup augmentation to drive our approach a step further. Experiments on CIFAR-10/100 and TinyImageNet demonstrate a robustness to label noise that substantially outperforms recent state-of-the-art. Source code is available at https://git.io/fjsvE

Accepted to ICML 2019

Cited by in corpus (54)

• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
• Sharpness-Aware Minimization for Efficiently Improving Generalization
• A Survey of Label-noise Representation Learning: Past, Present and Future
• Understanding and Improving Early Stopping for Learning with Noisy Labels
• Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
• Learning from Pixel-Level Label Noise: A New Perspective for Semi-Supervised Semantic Segmentation
• MoPro: Webly Supervised Learning with Momentum Prototypes
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• Uncertainty Based Detection and Relabeling of Noisy Image Labels
• Meta Transition Adaptation for Robust Deep Learning with Noisy Labels
• One-Step Abductive Multi-Target Learning with Diverse Noisy Samples and Its Application to Tumour Segmentation for Breast Cancer
• Variational Rectification Inference for Learning with Noisy Labels
• Towards Noise-resistant Object Detection with Noisy Annotations
• Learning From Long-Tailed Data With Noisy Labels
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learning
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Improving Unsupervised Image Clustering With Robust Learning
• Validation of the Practicability of Logical Assessment Formula for Evaluations with Inaccurate Ground-Truth Labels: An Application Study on Tumour Segmentation for Breast Cancer
• Derivative Manipulation for General Example Weighting
• Learning Fast Sample Re-weighting Without Reward Data
• MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation
• Faster Meta Update Strategy for Noise-Robust Deep Learning
• Confidence Adaptive Regularization for Deep Learning with Noisy Labels
• Countering Noisy Labels By Learning From Auxiliary Clean Labels
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels
• Unified Robust Training for Graph NeuralNetworks against Label Noise
• Adaptive Sample Selection for Robust Learning under Label Noise
• Towards Robustness to Label Noise in Text Classification via Noise Modeling
• Which Strategies Matter for Noisy Label Classification? Insight into Loss and Uncertainty
• Learning to Combat Noisy Labels via Classification Margins
• Co-matching: Combating Noisy Labels by Augmentation Anchoring
• NGC: A Unified Framework for Learning with Open-World Noisy Data
• How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning
• Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awareness
• Combating Label Noise With A General Surrogate Model For Sample Selection
• Transform consistency for learning with noisy labels
• Typing Errors in Factual Knowledge Graphs: Severity and Possible Ways Out
• CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
• Constrained Instance and Class Reweighting for Robust Learning under Label Noise
• A Survey on Deep Learning with Noisy Labels: How to train your model when you cannot trust on the annotations?
• Multiplicative Reweighting for Robust Neural Network Optimization
• DST: Data Selection and joint Training for Learning with Noisy Labels
• SENT: Sentence-level Distant Relation Extraction via Negative Training
• INN: A Method Identifying Clean-annotated Samples via Consistency Effect in Deep Neural Networks
• Mitigating Memorization in Sample Selection for Learning with Noisy Labels
• Alleviating Noisy-label Effects in Image Classification via Probability Transition Matrix
• Temporal-aware Language Representation Learning From Crowdsourced Labels
• Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction
• Knowledge Distillation with Noisy Labels for Natural Language Understanding