Learning from Noisy Labels with Distillation

Abstract

The ability of learning from noisy labels is very useful in many visual recognition tasks, as a vast amount of data with noisy labels are relatively easy to obtain. Traditionally, the label noises have been treated as statistical outliers, and approaches such as importance re-weighting and bootstrap have been proposed to alleviate the problem. According to our observation, the real-world noisy labels exhibit multi-mode characteristics as the true labels, rather than behaving like independent random outliers. In this work, we propose a unified distillation framework to use side information, including a small clean dataset and label relations in knowledge graph, to "hedge the risk" of learning from noisy labels. Furthermore, unlike the traditional approaches evaluated based on simulated label noises, we propose a suite of new benchmark datasets, in Sports, Species and Artifacts domains, to evaluate the task of learning from noisy labels in the practical setting. The empirical study demonstrates the effectiveness of our proposed method in all the domains.

References in corpus (3)

• Distilling the Knowledge in a Neural Network
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Training Convolutional Networks with Noisy Labels

Cited by in corpus (32)

• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Dimensionality-Driven Learning with Noisy Labels
• Masking: A New Perspective of Noisy Supervision
• How does Disagreement Help Generalization against Label Corruption?
• Label Refinery: Improving ImageNet Classification through Label Progression
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise
• Iterative Learning with Open-set Noisy Labels
• Learning with Bounded Instance- and Label-dependent Label Noise
• A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels
• Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark
• Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News
• Learning from Noisy Anchors for One-stage Object Detection
• DeepGUM: Learning Deep Robust Regression with a Gaussian-Uniform Mixture Model
• Less is More: Learning Highlight Detection from Video Duration
• The Resistance to Label Noise in K-NN and DNN Depends on its Concentration
• Weakly Supervised Learning with Side Information for Noisy Labeled Images
• Deep Learning from Noisy Image Labels with Quality Embedding
• Safeguarded Dynamic Label Regression for Generalized Noisy Supervision
• Graph Distillation for Action Detection with Privileged Modalities
• Limited Gradient Descent: Learning With Noisy Labels
• Learning from Noisy Labels with Noise Modeling Network
• Learning from a Lightweight Teacher for Efficient Knowledge Distillation
• Alternating Loss Correction for Preterm-Birth Prediction from EHR Data with Noisy Labels
• Multiplicative Reweighting for Robust Neural Network Optimization
• Do We Really Need Gold Samples for Sample Weighting Under Label Noise?
• Investigating naturalistic hand movements by behavior mining in long-term video and neural recordings
• Robust Temporal Ensembling for Learning with Noisy Labels
• Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression
• Task-Driven Data Verification via Gradient Descent