Calibrating Class Activation Maps for Long-Tailed Visual Recognition

Abstract

Real-world visual recognition problems often exhibit long-tailed distributions, where the amount of data for learning in different categories shows significant imbalance. Standard classification models learned on such data distribution often make biased predictions towards the head classes while generalizing poorly to the tail classes. In this paper, we present two effective modifications of CNNs to improve network learning from long-tailed distribution. First, we present a Class Activation Map Calibration (CAMC) module to improve the learning and prediction of network classifiers, by enforcing network prediction based on important image regions. The proposed CAMC module highlights the correlated image regions across data and reinforces the representations in these areas to obtain a better global representation for classification. Furthermore, we investigate the use of normalized classifiers for representation learning in long-tailed problems. Our empirical study demonstrates that by simply scaling the outputs of the classifier with an appropriate scalar, we can effectively improve the classification accuracy on tail classes without losing the accuracy of head classes. We conduct extensive experiments to validate the effectiveness of our design and we set new state-of-the-art performance on five benchmarks, including ImageNet-LT, Places-LT, iNaturalist 2018, CIFAR10-LT, and CIFAR100-LT.

References in corpus (20)

• SMOTE: Synthetic Minority Over-sampling Technique
• Prototypical Networks for Few-shot Learning
• mixup: Beyond Empirical Risk Minimization
• Learning to Reweight Examples for Robust Deep Learning
• Manifold Mixup: Better Representations by Interpolating Hidden States
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• Decoupling Representation and Classifier for Long-Tailed Recognition
• A Closer Look at Few-shot Classification
• Remix: Rebalanced Mixup
• Long-tail learning via logit adjustment
• DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning
• Conditional Gaussian Distribution Learning for Open Set Recognition
• Striking the Right Balance with Uncertainty
• CycleSegNet: Object Co-segmentation with Cycle Refinement and Region Correspondence
• Feature Space Augmentation for Long-Tailed Data
• M2IOSR: Maximal Mutual Information Open Set Recognition
• Open Set Recognition with Conditional Probabilistic Generative Models
• Imbalanced Continual Learning with Partitioning Reservoir Sampling
• Seesaw Loss for Long-Tailed Instance Segmentation