Label Noise Types and Their Effects on Deep Learning

Abstract

The recent success of deep learning is mostly due to the availability of big datasets with clean annotations. However, gathering a cleanly annotated dataset is not always feasible due to practical challenges. As a result, label noise is a common problem in datasets, and numerous methods to train deep neural networks in the presence of noisy labels are proposed in the literature. These methods commonly use benchmark datasets with synthetic label noise on the training set. However, there are multiple types of label noise, and each of them has its own characteristic impact on learning. Since each work generates a different kind of label noise, it is problematic to test and compare those algorithms in the literature fairly. In this work, we provide a detailed analysis of the effects of different kinds of label noise on learning. Moreover, we propose a generic framework to generate feature-dependent label noise, which we show to be the most challenging case for learning. Our proposed method aims to emphasize similarities among data instances by sparsely distributing them in the feature domain. By this approach, samples that are more likely to be mislabeled are detected from their softmax probabilities, and their labels are flipped to the corresponding class. The proposed method can be applied to any clean dataset to synthesize feature-dependent noisy labels. For the ease of other researchers to test their algorithms with noisy labels, we share corrupted labels for the most commonly used benchmark datasets. Our code and generated noisy synthetic labels are available online.

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Understanding deep learning requires rethinking generalization
• Learning to Reweight Examples for Robust Deep Learning
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Training Convolutional Networks with Noisy Labels
• Deep Learning is Robust to Massive Label Noise
• A Closer Look at Memorization in Deep Networks
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Dimensionality-Driven Learning with Noisy Labels
• How does Disagreement Help Generalization against Label Corruption?
• Learning From Noisy Singly-labeled Data
• Probabilistic End-to-end Noise Correction for Learning with Noisy Labels
• SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels

Cited by in corpus (8)

• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Generalized Data Weighting via Class-level Gradient Manipulation
• Robust Deep Learning with Active Noise Cancellation for Spatial Computing
• Planet Four: A Neural Network's Search For Polar Spring-time Fans On Mars
• Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond
• On the Robustness of Monte Carlo Dropout Trained with Noisy Labels
• The Influence of Faulty Labels in Data Sets on Human Pose Estimation
• Deep Metric Learning with Locality Sensitive Angular Loss for Self-Correcting Source Separation of Neural Spiking Signals