Energy Aligning for Biased Models

Abstract

Training on class-imbalanced data usually results in biased models that tend to predict samples into the majority classes, which is a common and notorious problem. From the perspective of energy-based model, we demonstrate that the free energies of categories are aligned with the label distribution theoretically, thus the energies of different classes are expected to be close to each other when aiming for ``balanced'' performance. However, we discover a severe energy-bias phenomenon in the models trained on class-imbalanced dataset. To eliminate the bias, we propose a simple and effective method named Energy Aligning by merely adding the calculated shift scalars onto the output logits during inference, which does not require to (i) modify the network architectures, (ii) intervene the standard learning paradigm, (iii) perform two-stage training. The proposed algorithm is evaluated on two class imbalance-related tasks under various settings: class incremental learning and long-tailed recognition. Experimental results show that energy aligning can effectively alleviate class imbalance issue and outperform state-of-the-art methods on several benchmarks.

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• mixup: Beyond Empirical Risk Minimization
• A systematic study of the class imbalance problem in convolutional neural networks
• Focal Loss for Dense Object Detection
• Continual Learning Through Synaptic Intelligence
• Learning to Reweight Examples for Robust Deep Learning
• Three scenarios for continual learning
• Manifold Mixup: Better Representations by Interpolating Hidden States
• Energy-based Out-of-distribution Detection
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
• Class-Balanced Loss Based on Effective Number of Samples
• Incremental Classifier Learning with Generative Adversarial Networks
• Large-Scale Long-Tailed Recognition in an Open World
• M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning
• How to Train Your Energy-Based Model for Regression