Learning Fast Sample Re-weighting Without Reward Data

Abstract

Training sample re-weighting is an effective approach for tackling data biases such as imbalanced and corrupted labels. Recent methods develop learning-based algorithms to learn sample re-weighting strategies jointly with model training based on the frameworks of reinforcement learning and meta learning. However, depending on additional unbiased reward data is limiting their general applicability. Furthermore, existing learning-based sample re-weighting methods require nested optimizations of models and weighting parameters, which requires expensive second-order computation. This paper addresses these two problems and presents a novel learning-based fast sample re-weighting (FSR) method that does not require additional reward data. The method is based on two key ideas: learning from history to build proxy reward data and feature sharing to reduce the optimization cost. Our experiments show the proposed method achieves competitive results compared to state of the arts on label noise robustness and long-tailed recognition, and does so while achieving significantly improved training efficiency. The source code is publicly available at https://github.com/google-research/google-research/tree/master/ieg.

ICCV2021

References in corpus (7)

• DivideMix: Learning with Noisy Labels as Semi-supervised Learning
• Unsupervised Label Noise Modeling and Loss Correction
• Data Shapley: Equitable Valuation of Data for Machine Learning
• Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
• Learning Data Manipulation for Augmentation and Weighting
• Robust Inference via Generative Classifiers for Handling Noisy Labels
• Learning to Learn from Weak Supervision by Full Supervision