Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting

Abstract

Current deep neural networks (DNNs) can easily overfit to biased training data with corrupted labels or class imbalance. Sample re-weighting strategy is commonly used to alleviate this issue by designing a weighting function mapping from training loss to sample weight, and then iterating between weight recalculating and classifier updating. Current approaches, however, need manually pre-specify the weighting function as well as its additional hyper-parameters. It makes them fairly hard to be generally applied in practice due to the significant variation of proper weighting schemes relying on the investigated problem and training data. To address this issue, we propose a method capable of adaptively learning an explicit weighting function directly from data. The weighting function is an MLP with one hidden layer, constituting a universal approximator to almost any continuous functions, making the method able to fit a wide range of weighting functions including those assumed in conventional research. Guided by a small amount of unbiased meta-data, the parameters of the weighting function can be finely updated simultaneously with the learning process of the classifiers. Synthetic and real experiments substantiate the capability of our method for achieving proper weighting functions in class imbalance and noisy label cases, fully complying with the common settings in traditional methods, and more complicated scenarios beyond conventional cases. This naturally leads to its better accuracy than other state-of-the-art methods.

arXiv commentNeurIPS 2019

References in corpus (19)

• SMOTE: Synthetic Minority Over-sampling Technique
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• A systematic study of the class imbalance problem in convolutional neural networks
• Learning to Reweight Examples for Robust Deep Learning
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• A Closer Look at Memorization in Deep Networks
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• Dimensionality-Driven Learning with Noisy Labels
• Class-Balanced Loss Based on Effective Number of Samples
• Joint Optimization Framework for Learning with Noisy Labels
• Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization
• Learning to Teach with Dynamic Loss Functions
• Learning from Noisy Labels with Distillation
• Small Sample Learning in Big Data Era
• Learning From Noisy Large-Scale Datasets With Minimal Supervision
• Learning to Teach
• Learning to Learn from Weak Supervision by Full Supervision
• Gradient Agreement as an Optimization Objective for Meta-Learning

Cited by in corpus (33)

• Decoupling Representation and Classifier for Long-Tailed Recognition
• Contrastive Meta Learning with Behavior Multiplicity for Recommendation
• A Survey on Long-Tailed Visual Recognition
• Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training
• Meta Balanced Network for Fair Face Recognition
• Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study
• ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learning
• Class-Difficulty Based Methods for Long-Tailed Visual Recognition
• Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher
• On Distinctive Image Captioning via Comparing and Reweighting
• Predicate correlation learning for scene graph generation
• HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks
• TGDM: Target Guided Dynamic Mixup for Cross-Domain Few-Shot Learning
• Equalization Loss for Long-Tailed Object Recognition
• Communication-Efficient Robust Federated Learning with Noisy Labels
• Learning Imbalanced Datasets with Maximum Margin Loss
• MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels
• A dual-branch model with inter- and intra-branch contrastive loss for long-tailed recognition
• Dirichlet-Based Prediction Calibration for Learning with Noisy Labels
• Variational Rectification Inference for Learning with Noisy Labels
• Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition
• Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection
• Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
• Adapting Triplet Importance of Implicit Feedback for Personalized Recommendation
• Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classification
• Self-balanced Learning For Domain Generalization
• Annotation-Free Human Sketch Quality Assessment
• Meta Corrupted Pixels Mining for Medical Image Segmentation
• Energy Aligning for Biased Models
• Combating Label Noise With A General Surrogate Model For Sample Selection
• Simple and Effective Input Reformulations for Translation
• Biquality Learning: a Framework to Design Algorithms Dealing with Closed-Set Distribution Shifts
• Learning advisor networks for noisy image classification