Combating Label Noise in Deep Learning Using Abstention

Abstract

We introduce a novel method to combat label noise when training deep neural networks for classification. We propose a loss function that permits abstention during training thereby allowing the DNN to abstain on confusing samples while continuing to learn and improve classification performance on the non-abstained samples. We show how such a deep abstaining classifier (DAC) can be used for robust learning in the presence of different types of label noise. In the case of structured or systematic label noise -- where noisy training labels or confusing examples are correlated with underlying features of the data-- training with abstention enables representation learning for features that are associated with unreliable labels. In the case of unstructured (arbitrary) label noise, abstention during training enables the DAC to be used as an effective data cleaner by identifying samples that are likely to have label noise. We provide analytical results on the loss function behavior that enable dynamic adaption of abstention rates based on learning progress during training. We demonstrate the utility of the deep abstaining classifier for various image classification tasks under different types of label noise; in the case of arbitrary label noise, we show significant improvements over previously published results on multiple image benchmarks. Source code is available at https://github.com/thulas/dac-label-noise

arXiv commentICML 2019. Added source code link

Cited by in corpus (33)

• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• An Overview of Deep Semi-Supervised Learning
• LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
• Self-Adaptive Training: beyond Empirical Risk Minimization
• SELF: Learning to Filter Noisy Labels with Self-Ensembling
• DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction
• Learning with Feature-Dependent Label Noise: A Progressive Approach
• Outside the Box: Abstraction-Based Monitoring of Neural Networks
• Controlled abstention neural networks for identifying skillful predictions for regression problems
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Controlled abstention neural networks for identifying skillful predictions for classification problems
• Distributional Generalization: A New Kind of Generalization
• Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning
• Towards Robustness to Label Noise in Text Classification via Noise Modeling
• A Unified Plug-and-Play Framework for Effective Data Denoising and Robust Abstention
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• Learning to Combat Noisy Labels via Classification Margins
• Deep k-NN for Noisy Labels
• On the predictability in reversible steganography
• Locally Adaptive Label Smoothing for Predictive Churn
• One for More: Selecting Generalizable Samples for Generalizable ReID Model
• Differentiable Learning Under Triage
• Know What Not To Know: Users' Perception of Abstaining Classifiers
• Ghost Loss to Question the Reliability of Training Data
• Multiplicative Reweighting for Robust Neural Network Optimization
• Reinforcement Learning Under Algorithmic Triage
• Classification Under Human Assistance
• Split-kl and PAC-Bayes-split-kl Inequalities for Ternary Random Variables
• Robust Temporal Ensembling for Learning with Noisy Labels
• Knowledge Distillation with Noisy Labels for Natural Language Understanding
• RISAN: Robust Instance Specific Abstention Network
• Regression Under Human Assistance
• Compensation Learning