Balanced Meta-Softmax for Long-Tailed Visual Recognition

Abstract

Deep classifiers have achieved great success in visual recognition. However, real-world data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this paper, we show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. Theoretically, we derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound. In addition, we introduce Balanced Meta-Softmax, applying a complementary Meta Sampler to estimate the optimal class sample rate and further improve long-tailed learning. In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.

arXiv commentNeurIPS 2020 camera-ready; Code available at https://github.com/jiawei-ren/BalancedMetaSoftmax

References in corpus (8)

• SMOTE: Synthetic Minority Over-sampling Technique
• Distributed Representations of Words and Phrases and their Compositionality
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A systematic study of the class imbalance problem in convolutional neural networks
• Learning to Reweight Examples for Robust Deep Learning
• Generalized Inner Loop Meta-Learning
• BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition
• Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning

Cited by in corpus (21)

• A Survey on Long-Tailed Visual Recognition
• FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
• A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications
• Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for Python
• Feature Fusion from Head to Tail for Long-Tailed Visual Recognition
• Hierarchical Memory Learning for Fine-Grained Scene Graph Generation
• Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning
• Toward Degree Bias in Embedding-Based Knowledge Graph Completion
• Inverse Image Frequency for Long-tailed Image Recognition
• Balanced softmax cross-entropy for incremental learning with and without memory
• FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognition
• Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition
• MSE Loss with Outlying Label for Imbalanced Classification
• Compound Batch Normalization for Long-tailed Image Classification
• Towards Overcoming False Positives in Visual Relationship Detection
• Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Tasks
• Investigating Group Distributionally Robust Optimization for Deep Imbalanced Learning: A Case Study of Binary Tabular Data Classification
• Balanced Representation Learning for Long-tailed Skeleton-based Action Recognition
• SMCL: Saliency Masked Contrastive Learning for Long-tailed Recognition
• Calibrating Class Activation Maps for Long-Tailed Visual Recognition
• LAE : Long-tailed Age Estimation