MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
arXiv:2012.05273
Abstract
Deep neural networks have been shown to easily overfit to biased training data with label noise or class imbalance. Meta-learning algorithms are commonly designed to alleviate this issue in the form of sample reweighting, by learning a meta weighting network that takes training losses as inputs to generate sample weights. In this paper, we advocate that choosing proper inputs for the meta weighting network is crucial for desired sample weights in a specific task, while training loss is not always the correct answer. In view of this, we propose a novel meta-learning algorithm, MetaInfoNet, which automatically learns effective representations as inputs for the meta weighting network by emphasizing task-related information with an information bottleneck strategy. Extensive experimental results on benchmark datasets with label noise or class imbalance validate that MetaInfoNet is superior to many state-of-the-art methods.
References in corpus (12)
- A systematic study of the class imbalance problem in convolutional neural networks
- The information bottleneck method
- Wide Residual Networks
- Training Deep Neural Networks on Noisy Labels with Bootstrapping
- A Closer Look at Memorization in Deep Networks
- Deep Variational Information Bottleneck
- Sensitivity and Generalization in Neural Networks: an Empirical Study
- Dimensionality-Driven Learning with Noisy Labels
- Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
- The Devil is in the Tails: Fine-grained Classification in the Wild
- SELF: Learning to Filter Noisy Labels with Self-Ensembling
- IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters