Constrained Instance and Class Reweighting for Robust Learning under Label Noise

Abstract

Deep neural networks have shown impressive performance in supervised learning, enabled by their ability to fit well to the provided training data. However, their performance is largely dependent on the quality of the training data and often degrades in the presence of noise. We propose a principled approach for tackling label noise with the aim of assigning importance weights to individual instances and class labels. Our method works by formulating a class of constrained optimization problems that yield simple closed form updates for these importance weights. The proposed optimization problems are solved per mini-batch which obviates the need of storing and updating the weights over the full dataset. Our optimization framework also provides a theoretical perspective on existing label smoothing heuristics for addressing label noise (such as label bootstrapping). We evaluate our method on several benchmark datasets and observe considerable performance gains in the presence of label noise.

27 pages, including Appendix

References in corpus (9)

• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Unsupervised Label Noise Modeling and Loss Correction
• Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
• Normalized Loss Functions for Deep Learning with Noisy Labels
• A Survey of Label-noise Representation Learning: Past, Present and Future
• Extended T: Learning with Mixed Closed-set and Open-set Noisy Labels
• Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond
• Multiplicative Reweighting for Robust Neural Network Optimization