Training Deep Neural Networks on Noisy Labels with Bootstrapping

Abstract

Current state-of-the-art deep learning systems for visual object recognition and detection use purely supervised training with regularization such as dropout to avoid overfitting. The performance depends critically on the amount of labeled examples, and in current practice the labels are assumed to be unambiguous and accurate. However, this assumption often does not hold; e.g. in recognition, class labels may be missing; in detection, objects in the image may not be localized; and in general, the labeling may be subjective. In this work we propose a generic way to handle noisy and incomplete labeling by augmenting the prediction objective with a notion of consistency. We consider a prediction consistent if the same prediction is made given similar percepts, where the notion of similarity is between deep network features computed from the input data. In experiments we demonstrate that our approach yields substantial robustness to label noise on several datasets. On MNIST handwritten digits, we show that our model is robust to label corruption. On the Toronto Face Database, we show that our model handles well the case of subjective labels in emotion recognition, achieving state-of-the- art results, and can also benefit from unlabeled face images with no modification to our method. On the ILSVRC2014 detection challenge data, we show that our approach extends to very deep networks, high resolution images and structured outputs, and results in improved scalable detection.

References in corpus (8)

• Improving neural networks by preventing co-adaptation of feature detectors
• Going Deeper with Convolutions
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Training Convolutional Networks with Noisy Labels
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Visualizing and Understanding Convolutional Networks
• DeepID-Net: multi-stage and deformable deep convolutional neural networks for object detection
• Analysis of Semi-Supervised Learning with the Yarowsky Algorithm

Cited by in corpus (192)

• Deep Facial Expression Recognition: A Survey
• Temporal Ensembling for Semi-Supervised Learning
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Learning to Reweight Examples for Robust Deep Learning
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Deep Learning is Robust to Massive Label Noise
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Dimensionality-Driven Learning with Noisy Labels
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
• Are Anchor Points Really Indispensable in Label-Noise Learning?
• Masking: A New Perspective of Noisy Supervision
• Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
• A Light CNN for Deep Face Representation with Noisy Labels
• Label Refinery: Improving ImageNet Classification through Label Progression
• Toward Controlled Generation of Text
• Hard Sample Aware Noise Robust Learning for Histopathology Image Classification
• Symmetric Cross Entropy for Robust Learning with Noisy Labels
• Decoupling "when to update" from "how to update"
• Robust Federated Learning with Noisy Labels
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix
• Learning to Fuse Things and Stuff
• Self-Adaptive Training: beyond Empirical Risk Minimization
• Webly Supervised Learning of Convolutional Networks
• Combating noisy labels by agreement: A joint training method with co-regularization
• ASAP: Architecture Search, Anneal and Prune
• Learning from Noisy Labels with Distillation
• Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels
• SELF: Learning to Filter Noisy Labels with Self-Ensembling
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
• Precise Proximal Femur Fracture Classification for Interactive Training and Surgical Planning
• EEG-Based Emotion Recognition Using Regularized Graph Neural Networks
• Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentation
• Weakly Supervised Vessel Segmentation in X-ray Angiograms by Self-Paced Learning from Noisy Labels with Suggestive Annotation
• Identifying Mislabeled Data using the Area Under the Margin Ranking
• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters
• Data Distillation: Towards Omni-Supervised Learning
• An Entropic Optimal Transport Loss for Learning Deep Neural Networks under Label Noise in Remote Sensing Images
• Label Noise Types and Their Effects on Deep Learning
• FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
• FINE Samples for Learning with Noisy Labels
• Unsupervised Domain Adaptation of Black-Box Source Models
• Confidence Scores Make Instance-dependent Label-noise Learning Possible
• Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher
• Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs
• Confidence Regularized Self-Training
• L_DMI: An Information-theoretic Noise-robust Loss Function
• Iterative Learning with Open-set Noisy Labels
• Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
• Towards Understanding Label Smoothing
• Provably End-to-end Label-Noise Learning without Anchor Points
• Learning with Instance-Dependent Label Noise: A Sample Sieve Approach
• Smooth Neighbors on Teacher Graphs for Semi-supervised Learning
• Uncertainty Based Detection and Relabeling of Noisy Image Labels
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
• Gunrock 2.0: A User Adaptive Social Conversational System
• Learning Video Object Segmentation from Unlabeled Videos
• Learning Deep Networks from Noisy Labels with Dropout Regularization
• MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels
• Does label smoothing mitigate label noise?
• SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
• NRGNN: Learning a Label Noise-Resistant Graph Neural Network on Sparsely and Noisily Labeled Graphs
• One-Step Abductive Multi-Target Learning with Diverse Noisy Samples and Its Application to Tumour Segmentation for Breast Cancer
• How does Early Stopping Help Generalization against Label Noise?
• Robust Learning Under Label Noise With Iterative Noise-Filtering
• Dirichlet-Based Prediction Calibration for Learning with Noisy Labels
• Deep Self-Learning From Noisy Labels
• Segmentation-Aware Image Denoising without Knowing True Segmentation
• CIA-Net: Robust Nuclei Instance Segmentation with Contour-aware Information Aggregation
• Variational Rectification Inference for Learning with Noisy Labels
• Handling Noisy Labels via One-Step Abductive Multi-Target Learning and Its Application to Helicobacter Pylori Segmentation
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks
• Large Margin Deep Networks for Classification
• Narrowing the Gap: Improved Detector Training with Noisy Location Annotations
• A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
• Breast Mass Detection with Faster R-CNN: On the Feasibility of Learning from Noisy Annotations
• Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification
• Improving Multi-Person Pose Estimation using Label Correction
• OMNIA Faster R-CNN: Detection in the wild through dataset merging and soft distillation
• DataCLUE: A Benchmark Suite for Data-centric NLP
• A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels
• Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels
• Searching to Exploit Memorization Effect in Learning from Corrupted Labels
• Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing
• In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation
• Validation of the Practicability of Logical Assessment Formula for Evaluations with Inaccurate Ground-Truth Labels: An Application Study on Tumour Segmentation for Breast Cancer
• A Survey on Curriculum Learning
• Coresets for Robust Training of Neural Networks against Noisy Labels
• Discriminative Self-training for Punctuation Prediction
• Bootstrapping Neural Processes
• Robust Training with Ensemble Consensus
• Derivative Manipulation for General Example Weighting
• Task Agnostic Robust Learning on Corrupt Outputs by Correlation-Guided Mixture Density Networks
• A Two-Stream Mutual Attention Network for Semi-supervised Biomedical Segmentation with Noisy Labels
• Label-Noise Robust Generative Adversarial Networks
• Fidelity-Weighted Learning
• Adaptive Sample Selection for Robust Learning under Label Noise
• Semantic-embedded Similarity Prototype for Scene Recognition
• Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning
• Towards Robustness to Label Noise in Text Classification via Noise Modeling
• On the Convergence of A Family of Robust Losses for Stochastic Gradient Descent
• Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation
• Improving Generalization by Controlling Label-Noise Information in Neural Network Weights
• Adaptive Early-Learning Correction for Segmentation from Noisy Annotations
• Attend in groups: a weakly-supervised deep learning framework for learning from web data
• Exploiting Web Images for Weakly Supervised Object Detection
• Less is More: Learning Highlight Detection from Video Duration
• Weakly Supervised Learning with Side Information for Noisy Labeled Images
• Robust and On-the-fly Dataset Denoising for Image Classification
• A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning
• Collaborative Label Correction via Entropy Thresholding
• Improving Distantly Supervised Relation Extraction with Neural Noise Converter and Conditional Optimal Selector
• Imperfect Segmentation Labels: How Much Do They Matter?
• A Novel Self-Supervised Re-labeling Approach for Training with Noisy Labels
• Self-Supervised Place Recognition by Refining Temporal and Featural Pseudo Labels from Panoramic Data
• Deep Bilevel Learning
• Deep Learning from Small Amount of Medical Data with Noisy Labels: A Meta-Learning Approach
• MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
• Learning from Noisy Web Data with Category-level Supervision
• Conservative Predictions on Noisy Financial Data
• Learning to Combat Noisy Labels via Classification Margins
• KNN-enhanced Deep Learning Against Noisy Labels
• A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• Robust Deep Learning with Active Noise Cancellation for Spatial Computing
• Robust Deep Graph Based Learning for Binary Classification
• Automated Inline Analysis of Myocardial Perfusion MRI with Deep Learning
• Co-matching: Combating Noisy Labels by Augmentation Anchoring
• Joint Negative and Positive Learning for Noisy Labels
• NLNL: Negative Learning for Noisy Labels
• Safeguarded Dynamic Label Regression for Generalized Noisy Supervision
• Consistency Regularization Can Improve Robustness to Label Noise
• Learning Deep Neural Networks under Agnostic Corrupted Supervision
• A Practical Framework for Relation Extraction with Noisy Labels Based on Doubly Transitional Loss
• Learning from Noisy Labels with Noise Modeling Network
• Preserving Differential Privacy in Convolutional Deep Belief Networks
• Temporal Self-Ensembling Teacher for Semi-Supervised Object Detection
• Data-driven Meta-set Based Fine-Grained Visual Classification
• Improving Medical Annotation Quality to Decrease Labeling Burden Using Stratified Noisy Cross-Validation
• Multi-source weak supervision for saliency detection
• Random Bundle: Brain Metastases Segmentation Ensembling through Annotation Randomization
• Product Image Recognition with Guidance Learning and Noisy Supervision
• Adversarial Bootstrapping for Dialogue Model Training
• Limited Gradient Descent: Learning With Noisy Labels
• GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
• Constrained Instance and Class Reweighting for Robust Learning under Label Noise
• CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
• Reliable Label Bootstrapping for Semi-Supervised Learning
• Investigation of Training Label Error Impact on RNN-T
• Calibrated Top-1 Uncertainty estimates for classification by score based models
• Graph convolutional networks for learning with few clean and many noisy labels
• Understanding Patch-Based Learning by Explaining Predictions
• Ranking-Based Autoencoder for Extreme Multi-label Classification
• Decoupled Gradient Harmonized Detector for Partial Annotation: Application to Signet Ring Cell Detection
• Deep Learning for Earth Image Segmentation based on Imperfect Polyline Labels with Annotation Errors
• Dense Regression Activation Maps For Lesion Segmentation in CT scans of COVID-19 patients
• Meta Label Correction for Noisy Label Learning
• Bootstrapping Deep Neural Networks from Approximate Image Processing Pipelines
• Robust Temporal Ensembling for Learning with Noisy Labels
• Noise Robust Generative Adversarial Networks
• Learning to Reweight with Deep Interactions
• Using Under-trained Deep Ensembles to Learn Under Extreme Label Noise
• Improving Generalization of Deep Fault Detection Models in the Presence of Mislabeled Data
• Learning with Noisy Labels for Sentence-level Sentiment Classification
• Neural Bootstrapper
• MagicPai at SemEval-2021 Task 7: Method for Detecting and Rating Humor Based on Multi-Task Adversarial Training
• Learning from Web Data with Self-Organizing Memory Module
• MULTIMODAL ANALYSIS: Informed content estimation and audio source separation
• A Mathematical Foundation for Robust Machine Learning based on Bias-Variance Trade-off
• Compensation Learning
• Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis
• Which Samples Should be Learned First: Easy or Hard?
• Alleviating Noisy-label Effects in Image Classification via Probability Transition Matrix
• Adopting Robustness and Optimality in Fitting and Learning
• Skeptical Deep Learning with Distribution Correction
• Learning advisor networks for noisy image classification
• Training Convolutional Networks with Web Images
• Mitigating Memorization in Sample Selection for Learning with Noisy Labels
• Suppressing Mislabeled Data via Grouping and Self-Attention
• Simpler Certified Radius Maximization by Propagating Covariances
• Towards Bottom-Up Analysis of Social Food
• Joint Deep Learning of Facial Expression Synthesis and Recognition
• Disentanglement based Active Learning