Learning Adaptive Loss for Robust Learning with Noisy Labels
arXiv:2002.06482
Abstract
Robust loss minimization is an important strategy for handling robust learning issue on noisy labels. Current robust loss functions, however, inevitably involve hyperparameter(s) to be tuned, manually or heuristically through cross validation, which makes them fairly hard to be generally applied in practice. Besides, the non-convexity brought by the loss as well as the complicated network architecture makes it easily trapped into an unexpected solution with poor generalization capability. To address above issues, we propose a meta-learning method capable of adaptively learning hyperparameter in robust loss functions. Specifically, through mutual amelioration between robust loss hyperparameter and network parameters in our method, both of them can be simultaneously finely learned and coordinated to attain solutions with good generalization capability. Four kinds of SOTA robust loss functions are attempted to be integrated into our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its accuracy and generalization performance, as compared with conventional hyperparameter tuning strategy, even with carefully tuned hyperparameters.
10pages
References in corpus (10)
- Practical Bayesian Optimization of Machine Learning Algorithms
- Theoretical Models of Learning to Learn
- Noise Tolerance under Risk Minimization
- Making Risk Minimization Tolerant to Label Noise
- Curriculum Loss: Robust Learning and Generalization against Label Corruption
- Learning to Teach with Dynamic Loss Functions
- Small Sample Learning in Big Data Era
- L_DMI: An Information-theoretic Noise-robust Loss Function
- Learning Surrogate Losses
- Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment