Dirichlet-Based Prediction Calibration for Learning with Noisy Labels

Abstract

Learning with noisy labels can significantly hinder the generalization performance of deep neural networks (DNNs). Existing approaches address this issue through loss correction or example selection methods. However, these methods often rely on the model's predictions obtained from the softmax function, which can be over-confident and unreliable. In this study, we identify the translation invariance of the softmax function as the underlying cause of this problem and propose the \textit{Dirichlet-based Prediction Calibration} (DPC) method as a solution. Our method introduces a calibrated softmax function that breaks the translation invariance by incorporating a suitable constant in the exponent term, enabling more reliable model predictions. To ensure stable model training, we leverage a Dirichlet distribution to assign probabilities to predicted labels and introduce a novel evidence deep learning (EDL) loss. The proposed loss function encourages positive and sufficiently large logits for the given label, while penalizing negative and small logits for other labels, leading to more distinct logits and facilitating better example selection based on a large-margin criterion. Through extensive experiments on diverse benchmark datasets, we demonstrate that DPC achieves state-of-the-art performance. The code is available at https://github.com/chenchenzong/DPC.

References in corpus (26)

• mixup: Beyond Empirical Risk Minimization
• On Calibration of Modern Neural Networks
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Evidential Deep Learning to Quantify Classification Uncertainty
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• WebVision Database: Visual Learning and Understanding from Web Data
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Dimensionality-Driven Learning with Noisy Labels
• Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
• Normalized Loss Functions for Deep Learning with Noisy Labels
• Decoupling "when to update" from "how to update"
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• Self-Adaptive Training: beyond Empirical Risk Minimization
• Probabilistic End-to-end Noise Correction for Learning with Noisy Labels
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Understanding and Improving Early Stopping for Learning with Noisy Labels
• Robust Training under Label Noise by Over-parameterization
• UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning
• Dirichlet-based Uncertainty Calibration for Active Domain Adaptation
• Noise-Robust Bidirectional Learning with Dynamic Sample Reweighting

Cited by in corpus (1)

• Bidirectional Uncertainty-Based Active Learning for Open Set Annotation