Adaptive Sample Selection for Robust Learning under Label Noise

Abstract

Deep Neural Networks (DNNs) have been shown to be susceptible to memorization or overfitting in the presence of noisily-labelled data. For the problem of robust learning under such noisy data, several algorithms have been proposed. A prominent class of algorithms rely on sample selection strategies wherein, essentially, a fraction of samples with loss values below a certain threshold are selected for training. These algorithms are sensitive to such thresholds, and it is difficult to fix or learn these thresholds. Often, these algorithms also require information such as label noise rates which are typically unavailable in practice. In this paper, we propose an adaptive sample selection strategy that relies only on batch statistics of a given mini-batch to provide robustness against label noise. The algorithm does not have any additional hyperparameters for sample selection, does not need any information on noise rates and does not need access to separate data with clean labels. We empirically demonstrate the effectiveness of our algorithm on benchmark datasets.

arXiv commentAccepted at WACV 2023

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• mixup: Beyond Empirical Risk Minimization
• Understanding deep learning requires rethinking generalization
• Learning to Reweight Examples for Robust Deep Learning
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• A Closer Look at Memorization in Deep Networks
• Dimensionality-Driven Learning with Noisy Labels
• Unsupervised Label Noise Modeling and Loss Correction
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• Normalized Loss Functions for Deep Learning with Noisy Labels
• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• Combating noisy labels by agreement: A joint training method with co-regularization
• On Symmetric Losses for Learning from Corrupted Labels
• Error-Bounded Correction of Noisy Labels
• Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise