On Symmetric Losses for Learning from Corrupted Labels

Abstract

This paper aims to provide a better understanding of a symmetric loss. First, we emphasize that using a symmetric loss is advantageous in the balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization from corrupted labels. Second, we prove general theoretical properties of symmetric losses, including a classification-calibration condition, excess risk bound, conditional risk minimizer, and AUC-consistency condition. Third, since all nonnegative symmetric losses are non-convex, we propose a convex barrier hinge loss that benefits significantly from the symmetric condition, although it is not symmetric everywhere. Finally, we conduct experiments to validate the relevance of the symmetric condition.

arXiv commentICML2019 with minor typo fixes

References in corpus (9)

• On the Convergence of Adam and Beyond
• Noise Tolerance under Risk Minimization
• Making Risk Minimization Tolerant to Label Noise
• Learning from Complementary Labels
• Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
• On Symmetric Losses for Learning from Corrupted Labels
• Semi-Supervised AUC Optimization based on Positive-Unlabeled Learning
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Sparse Robust Classification via the Kernel Mean

Cited by in corpus (23)

• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates
• On Symmetric Losses for Learning from Corrupted Labels
• Confidence Scores Make Instance-dependent Label-noise Learning Possible
• Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher
• Noise-tolerant fair classification
• Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Complementary-Label Learning for Arbitrary Losses and Models
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Derivative Manipulation for General Example Weighting
• Classification with Rejection Based on Cost-sensitive Classification
• Combining Self-Supervised and Supervised Learning with Noisy Labels
• Adaptive Sample Selection for Robust Learning under Label Noise
• Robust Imitation Learning from Noisy Demonstrations
• When Optimizing $f$-divergence is Robust with Label Noise
• Learning from Label Proportions: A Mutual Contamination Framework
• Importance Reweighting for Biquality Learning
• Robust Deep Learning with Active Noise Cancellation for Spatial Computing
• Robustness and Reliability When Training With Noisy Labels
• A Symmetric Loss Perspective of Reliable Machine Learning
• High-level Prior-based Loss Functions for Medical Image Segmentation: A Survey
• Do We Really Need Gold Samples for Sample Weighting Under Label Noise?