Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates

Abstract

Learning with noisy labels is a common challenge in supervised learning. Existing approaches often require practitioners to specify noise rates, i.e., a set of parameters controlling the severity of label noises in the problem, and the specifications are either assumed to be given or estimated using additional steps. In this work, we introduce a new family of loss functions that we name as peer loss functions, which enables learning from noisy labels and does not require a priori specification of the noise rates. Peer loss functions work within the standard empirical risk minimization (ERM) framework. We show that, under mild conditions, performing ERM with peer loss functions on the noisy dataset leads to the optimal or a near-optimal classifier as if performing ERM over the clean training data, which we do not have access to. We pair our results with an extensive set of experiments. Peer loss provides a way to simplify model development when facing potentially noisy training labels, and can be promoted as a robust candidate loss function in such situations.

arXiv commentICML 2020. Corrected typos in the Appendix

References in corpus (9)

• mixup: Beyond Empirical Risk Minimization
• Classification with Noisy Labels by Importance Reweighting
• Noise Tolerance under Risk Minimization
• Making Risk Minimization Tolerant to Label Noise
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• On Symmetric Losses for Learning from Corrupted Labels
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• L_DMI: An Information-theoretic Noise-robust Loss Function
• Sparse Robust Classification via the Kernel Mean

Cited by in corpus (4)

• One-Step Abductive Multi-Target Learning with Diverse Noisy Samples and Its Application to Tumour Segmentation for Breast Cancer
• Variational Rectification Inference for Learning with Noisy Labels
• "Is not the truth the truth?": Analyzing the Impact of User Validations for Bus In/Out Detection in Smartphone-based Surveys
• Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond