Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization

Abstract

Many weakly supervised classification methods employ a noise transition matrix to capture the class-conditional label corruption. To estimate the transition matrix from noisy data, existing methods often need to estimate the noisy class-posterior, which could be unreliable due to the overconfidence of neural networks. In this work, we propose a theoretically grounded method that can estimate the noise transition matrix and learn a classifier simultaneously, without relying on the error-prone noisy class-posterior estimation. Concretely, inspired by the characteristics of the stochastic label corruption process, we propose total variation regularization, which encourages the predicted probabilities to be more distinguishable from each other. Under mild assumptions, the proposed method yields a consistent estimator of the transition matrix. We show the effectiveness of the proposed method through experiments on benchmark and real-world datasets.

arXiv commentICML 2021 camera-ready version

References in corpus (18)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• On Calibration of Modern Neural Networks
• Classification with Noisy Labels by Importance Reweighting
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• A Closer Look at Memorization in Deep Networks
• Are Anchor Points Really Indispensable in Label-Noise Learning?
• Masking: A New Perspective of Noisy Supervision
• Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• Combating Label Noise in Deep Learning Using Abstention
• Learning from Complementary Labels
• Mixture Proportion Estimation via Kernel Embedding of Distributions
• On Symmetric Losses for Learning from Corrupted Labels
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Provably End-to-end Label-Noise Learning without Anchor Points
• A Topological Filter for Learning with Label Noise

Cited by in corpus (3)

• Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives
• Variational Rectification Inference for Learning with Noisy Labels
• A Second-Order Approach to Learning with Instance-Dependent Label Noise