Combating Label Noise With A General Surrogate Model For Sample Selection

Abstract

Modern deep learning systems are data-hungry. Learning with web data is one of the feasible solutions, but will introduce label noise inevitably, which can hinder the performance of deep neural networks. Sample selection is an effective way to deal with label noise. The key is to separate clean samples based on some criterion. Previous methods pay more attention to the small loss criterion where small-loss samples are regarded as clean ones. Nevertheless, such a strategy relies on the learning dynamics of each data instance. Some noisy samples are still memorized due to frequently occurring corrupted learning patterns. To tackle this problem, a training-free surrogate model is preferred, freeing from the effect of memorization. In this work, we propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically. CLIP brings external knowledge to facilitate the selection of clean samples with its ability of text-image alignment. Furthermore, a margin adaptive loss is designed to regularize the selection bias introduced by CLIP, providing robustness to label noise. We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets. Our method achieves significant improvement without CLIP involved during the inference stage.

Accepted by IJCV 2024

References in corpus (20)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Learning Transferable Visual Models From Natural Language Supervision
• Classification with Noisy Labels by Importance Reweighting
• MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Unsupervised Label Noise Modeling and Loss Correction
• Part-dependent Label Noise: Towards Instance-dependent Label Noise
• CenterCLIP: Token Clustering for Efficient Text-Video Retrieval
• Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• DataComp: In search of the next generation of multimodal datasets
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations
• Instance-dependent Label-noise Learning under a Structural Causal Model
• Detecting Corrupted Labels Without Training a Model to Predict
• Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition
• GenKL: An Iterative Framework for Resolving Label Ambiguity and Label Non-conformity in Web Images Via a New Generalized KL Divergence