Decoupling Representation and Classifier for Long-Tailed Recognition

Abstract

The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but most of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification. Our code is available at https://github.com/facebookresearch/classifier-balancing.

References in corpus (7)

• SMOTE: Synthetic Minority Over-sampling Technique
• Distributed Representations of Words and Phrases and their Compositionality
• Prototypical Networks for Few-shot Learning
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Max-margin Class Imbalanced Learning with Gaussian Affinity
• A Study on Action Detection in the Wild

Cited by in corpus (34)

• Rethinking the Value of Labels for Improving Class-Imbalanced Learning
• A Survey on Long-Tailed Visual Recognition
• Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for Python
• Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study
• Feature Fusion from Head to Tail for Long-Tailed Visual Recognition
• Class-Difficulty Based Methods for Long-Tailed Visual Recognition
• Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective
• Hierarchical Memory Learning for Fine-Grained Scene Graph Generation
• Equalization Loss for Long-Tailed Object Recognition
• MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition
• Analysis of an adaptive lead weighted ResNet for multiclass classification of 12-lead ECGs
• ELF: An Early-Exiting Framework for Long-Tailed Classification
• Rethinking Re-Sampling in Imbalanced Semi-Supervised Learning
• Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation
• Self-Damaging Contrastive Learning
• Visual Relationship Detection using Scene Graphs: A Survey
• ROI-Aware Multiscale Cross-Attention Vision Transformer for Pest Image Identification
• ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shot
• Balanced-MixUp for Highly Imbalanced Medical Image Classification
• Classification Calibration for Long-tail Instance Segmentation
• Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection
• Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data
• Relation Modeling in Spatio-Temporal Action Localization
• Adaptive Parametric Activation: Unifying and Generalising Activation Functions Across Tasks
• Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
• The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation
• BeeTLe: An Imbalance-Aware Deep Sequence Model for Linear B-Cell Epitope Prediction and Classification with Logit-Adjusted Losses
• Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition
• AlphaNet: Improving Long-Tail Classification By Combining Classifiers
• Balanced Representation Learning for Long-tailed Skeleton-based Action Recognition
• A Linkage-based Doubly Imbalanced Graph Learning Framework for Face Clustering
• Calibrating Class Activation Maps for Long-Tailed Visual Recognition
• LAE : Long-tailed Age Estimation