Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise

Abstract

The growing importance of massive datasets used for deep learning makes robustness to label noise a critical property for classifiers to have. Sources of label noise include automatic labeling, non-expert labeling, and label corruption by data poisoning adversaries. Numerous previous works assume that no source of labels can be trusted. We relax this assumption and assume that a small subset of the training data is trusted. This enables substantial label corruption robustness performance gains. In addition, particularly severe label noise can be combated by using a set of trusted data with clean labels. We utilize trusted data by proposing a loss correction technique that utilizes trusted examples in a data-efficient manner to mitigate the effects of label noise on deep neural network classifiers. Across vision and natural language processing tasks, we experiment with various label noises at several strengths, and show that our method significantly outperforms existing methods.

arXiv commentNeurIPS 2018. PyTorch code available at https://github.com/mmazeika/glc

References in corpus (8)

• On Calibration of Modern Neural Networks
• Learning to Reweight Examples for Robust Deep Learning
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Aggregated Residual Transformations for Deep Neural Networks
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering
• Certified Defenses for Data Poisoning Attacks
• Learning from Noisy Labels with Distillation
• Learning from Binary Labels with Instance-Dependent Corruption

Cited by in corpus (21)

• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Masking: A New Perspective of Noisy Supervision
• Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
• Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• An Entropic Optimal Transport Loss for Learning Deep Neural Networks under Label Noise in Remote Sensing Images
• Learning with Weak Supervision for Email Intent Detection
• Reinforcement Learning with Perturbed Rewards
• Variational Rectification Inference for Learning with Noisy Labels
• On Universal Black-Box Domain Adaptation
• Robust and On-the-fly Dataset Denoising for Image Classification
• Robust Deep Graph Based Learning for Binary Classification
• A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
• NLNL: Negative Learning for Noisy Labels
• On the Robustness of Monte Carlo Dropout Trained with Noisy Labels
• Distilling effective supervision for robust medical image segmentation with noisy labels
• Combating Label Noise With A General Surrogate Model For Sample Selection
• Reliable Label Bootstrapping for Semi-Supervised Learning
• Do We Really Need Gold Samples for Sample Weighting Under Label Noise?
• Mitigating Memorization in Sample Selection for Learning with Noisy Labels
• Self-Training with Weak Supervision