Collaborative Label Correction via Entropy Thresholding
arXiv:2103.17008 · doi:10.1109/ICDM.2019.00179
Abstract
Deep neural networks (DNNs) have the capacity to fit extremely noisy labels nonetheless they tend to learn data with clean labels first and then memorize those with noisy labels. We examine this behavior in light of the Shannon entropy of the predictions and demonstrate the low entropy predictions determined by a given threshold are much more reliable as the supervision than the original noisy labels. It also shows the advantage in maintaining more training samples than previous methods. Then, we power this entropy criterion with the Collaborative Label Correction (CLC) framework to further avoid undesired local minimums of the single network. A range of experiments have been conducted on multiple benchmarks with both synthetic and real-world settings. Extensive results indicate that our CLC outperforms several state-of-the-art methods.
arXiv commentICDM 2019
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Sequence to Sequence Learning with Neural Networks
- Training Deep Neural Networks on Noisy Labels with Bootstrapping
- MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
- A Closer Look at Memorization in Deep Networks
- Masking: A New Perspective of Noisy Supervision
- Decoupling "when to update" from "how to update"
- Collaborative Learning for Deep Neural Networks