Feature Fusion from Head to Tail for Long-Tailed Visual Recognition

Abstract

The imbalanced distribution of long-tailed data presents a considerable challenge for deep learning models, as it causes them to prioritize the accurate classification of head classes but largely disregard tail classes. The biased decision boundary caused by inadequate semantic information in tail classes is one of the key factors contributing to their low recognition accuracy. To rectify this issue, we propose to augment tail classes by grafting the diverse semantic information from head classes, referred to as head-to-tail fusion (H2T). We replace a portion of feature maps from tail classes with those belonging to head classes. These fused features substantially enhance the diversity of tail classes. Both theoretical analysis and practical experimentation demonstrate that H2T can contribute to a more optimized solution for the decision boundary. We seamlessly integrate H2T in the classifier adjustment stage, making it a plug-and-play module. Its simplicity and ease of implementation allow for smooth integration with existing long-tailed recognition methods, facilitating a further performance boost. Extensive experiments on various long-tailed benchmarks demonstrate the effectiveness of the proposed H2T. The source code is available at https://github.com/Keke921/H2T.

arXiv commentAccepted to AAAI24, similar to the conference version. Add the supplementry

References in corpus (27)

• SMOTE: Synthetic Minority Over-sampling Technique
• mixup: Beyond Empirical Risk Minimization
• Learning to Reweight Examples for Robust Deep Learning
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• NormFace: L2 Hypersphere Embedding for Face Verification
• Manifold Mixup: Better Representations by Interpolating Hidden States
• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• CosFace: Large Margin Cosine Loss for Deep Face Recognition
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval
• Decoupling Representation and Classifier for Long-Tailed Recognition
• A Survey on Long-Tailed Visual Recognition
• Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules
• Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustment
• BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition
• Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective
• Deep Imbalanced Learning for Face Recognition and Attribute Prediction
• Long-tailed Recognition by Routing Diverse Distribution-Aware Experts
• Disentangling Label Distribution for Long-tailed Visual Recognition
• Deep Over-sampling Framework for Classifying Imbalanced Data
• M2m: Imbalanced Classification via Major-to-minor Translation
• MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition
• Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification
• Pure Noise to the Rescue of Insufficient Data: Improving Imbalanced Classification by Training on Random Noise Images
• ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shot
• Nested Collaborative Learning for Long-Tailed Visual Recognition
• Does Head Label Help for Long-Tailed Multi-Label Text Classification

Cited by in corpus (1)

• Long-Tailed Learning for Generalized Category Discovery