Making Risk Minimization Tolerant to Label Noise

Abstract

In many applications, the training data, from which one needs to learn a classifier, is corrupted with label noise. Many standard algorithms such as SVM perform poorly in presence of label noise. In this paper we investigate the robustness of risk minimization to label noise. We prove a sufficient condition on a loss function for the risk minimization under that loss to be tolerant to uniform label noise. We show that the $0-1$ loss, sigmoid loss, ramp loss and probit loss satisfy this condition though none of the standard convex loss functions satisfy it. We also prove that, by choosing a sufficiently large value of a parameter in the loss function, the sigmoid loss, ramp loss and probit loss can be made tolerant to non-uniform label noise also if we can assume the classes to be separable under noise-free data distribution. Through extensive empirical studies, we show that risk minimization under the $0-1$ loss, the sigmoid loss and the ramp loss has much better robustness to label noise when compared to the SVM algorithm.

References in corpus (2)

• Identifying Mislabeled Training Data
• Noise Tolerance under Risk Minimization

Cited by in corpus (52)

• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates
• On Symmetric Losses for Learning from Corrupted Labels
• Double Ramp Loss Based Reject Option Classifier
• An Entropic Optimal Transport Loss for Learning Deep Neural Networks under Label Noise in Remote Sensing Images
• Loss factorization, weakly supervised learning and label noise robustness
• Confidence Scores Make Instance-dependent Label-noise Learning Possible
• Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher
• Learning with Feature-Dependent Label Noise: A Progressive Approach
• Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
• Learning with Bounded Instance- and Label-dependent Label Noise
• Deep Self-Learning From Noisy Labels
• Learning Adaptive Loss for Robust Learning with Noisy Labels
• SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise
• Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Contrastive Learning Improves Model Robustness Under Label Noise
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels
• On the Robustness of Decision Tree Learning under Label Noise
• Classification with Rejection Based on Cost-sensitive Classification
• The Fisher-Rao Loss for Learning under Label Noise
• When Optimizing $f$-divergence is Robust with Label Noise
• Weakly Supervised Learning with Side Information for Noisy Labeled Images
• The Resistance to Label Noise in K-NN and DNN Depends on its Concentration
• A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptation
• Learnability with Indirect Supervision Signals
• Learning to Combat Noisy Labels via Classification Margins
• Joint Negative and Positive Learning for Noisy Labels
• NLNL: Negative Learning for Noisy Labels
• Cost Sensitive Learning in the Presence of Symmetric Label Noise
• Defending against substitute model black box adversarial attacks with the 01 loss
• Limited Gradient Descent: Learning With Noisy Labels
• On the transferability of adversarial examples between convex and 01 loss models
• Multiplicative Reweighting for Robust Neural Network Optimization
• Robustness and Reliability When Training With Noisy Labels
• Towards Robust Pattern Recognition: A Review
• Robust binary classification with the 01 loss
• On the Robustness of Average Losses for Partial-Label Learning
• Adma: A Flexible Loss Function for Neural Networks
• Noisy Labels for Weakly Supervised Gamma Hadron Classification
• Do We Really Need Gold Samples for Sample Weighting Under Label Noise?
• A Symmetric Loss Perspective of Reliable Machine Learning
• Temporal Calibrated Regularization for Robust Noisy Label Learning
• Towards adversarial robustness with 01 loss neural networks
• Label Noise Filtering Techniques to Improve Monotonic Classification
• Learning Only from Relevant Keywords and Unlabeled Documents
• Making Convex Loss Functions Robust to Outliers using $e$-Exponentiated Transformation
• A Non-Intrusive Correction Algorithm for Classification Problems with Corrupted Data
• Learning with Inadequate and Incorrect Supervision