Adaptive Sample Selection for Robust Learning under Label Noise

Abstract

Deep Neural Networks (DNNs) have been shown to be susceptible to memorization or overfitting in the presence of noisily-labelled data. For the problem of robust learning under such noisy data, several algorithms have been proposed. A prominent class of algorithms rely on sample selection strategies wherein, essentially, a fraction of samples with loss values below a certain threshold are selected for training. These algorithms are sensitive to such thresholds, and it is difficult to fix or learn these thresholds. Often, these algorithms also require information such as label noise rates which are typically unavailable in practice. In this paper, we propose an adaptive sample selection strategy that relies only on batch statistics of a given mini-batch to provide robustness against label noise. The algorithm does not have any additional hyperparameters for sample selection, does not need any information on noise rates and does not need access to separate data with clean labels. We empirically demonstrate the effectiveness of our algorithm on benchmark datasets.

Accepted at WACV 2023

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Understanding deep learning requires rethinking generalization
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• A Closer Look at Memorization in Deep Networks
• Unsupervised Label Noise Modeling and Loss Correction
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• Normalized Loss Functions for Deep Learning with Noisy Labels
• Combating noisy labels by agreement: A joint training method with co-regularization
• Error-Bounded Correction of Noisy Labels
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise