FINE Samples for Learning with Noisy Labels
arXiv:2102.11628
Abstract
Modern deep neural networks (DNNs) become frail when the datasets contain noisy (incorrect) class labels. Robust techniques in the presence of noisy labels can be categorized into two folds: developing noise-robust functions or using noise-cleansing methods by detecting the noisy data. Recently, noise-cleansing methods have been considered as the most competitive noisy-label learning algorithms. Despite their success, their noisy label detectors are often based on heuristics more than a theory, requiring a robust classifier to predict the noisy data with loss values. In this paper, we propose a novel detector for filtering label noise. Unlike most existing methods, we focus on each data's latent representation dynamics and measure the alignment between the latent distribution and each representation using the eigendecomposition of the data gram matrix. Our framework, coined as filtering noisy instances via their eigenvectors (FINE), provides a robust detector with derivative-free simple methods having theoretical guarantees. Under our framework, we propose three applications of the FINE: sample-selection approach, semi-supervised learning approach, and collaboration with noise-robust loss functions. Experimental results show that the proposed methods consistently outperform corresponding baselines for all three applications on various benchmark datasets.
arXiv commentThe Thirty-Fifth Annual Conference on Neural Information Processing Systems (NeurIPS 2021)
References in corpus (20)
- SMOTE: Synthetic Minority Over-sampling Technique
- mixup: Beyond Empirical Risk Minimization
- A systematic study of the class imbalance problem in convolutional neural networks
- A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
- Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
- Learning to Reweight Examples for Robust Deep Learning
- Training Deep Neural Networks on Noisy Labels with Bootstrapping
- DivideMix: Learning with Noisy Labels as Semi-supervised Learning
- A Closer Look at Memorization in Deep Networks
- Adversarial Filters of Dataset Biases
- How does Disagreement Help Generalization against Label Corruption?
- Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
- LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
- Leveraging Weakly Annotated Data for Fashion Image Retrieval and Label Prediction
- Robust Inference via Generative Classifiers for Handling Noisy Labels
- Identifying Mislabeled Data using the Area Under the Margin Ranking
- What Do Neural Networks Learn When Trained With Random Labels?
- A Topological Filter for Learning with Label Noise
- DST: Data Selection and joint Training for Learning with Noisy Labels