A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
arXiv:1909.09338
Abstract
Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance. To mitigate this issue, we provide a simple but effective baseline method that is robust to noisy labels, even with severe noise. Our objective involves a variance regularization term that implicitly penalizes the Jacobian norm of the neural network on the whole training set (including the noisy-labeled data), which encourages generalization and prevents overfitting to the corrupted labels. Experiments on both synthetically generated incorrect labels and realistic large-scale noisy datasets demonstrate that our approach achieves state-of-the-art performance with a high tolerance to severe noise.
References in corpus (19)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- Wide Residual Networks
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Temporal Ensembling for Semi-Supervised Learning
- Understanding deep learning requires rethinking generalization
- Learning to Reweight Examples for Robust Deep Learning
- Opening the Black Box of Deep Neural Networks via Information
- AutoAugment: Learning Augmentation Policies from Data
- Training Deep Neural Networks on Noisy Labels with Bootstrapping
- MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
- A Closer Look at Memorization in Deep Networks
- WebVision Database: Visual Learning and Understanding from Web Data
- Sensitivity and Generalization in Neural Networks: an Empirical Study
- Robust Large Margin Deep Neural Networks
- Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
- Auxiliary Image Regularization for Deep CNNs with Noisy Labels
- Learning from Binary Labels with Instance-Dependent Corruption