Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect
arXiv:2009.12991
Abstract
As the class size grows, maintaining a balanced dataset across many classes is challenging because the data are long-tailed in nature; it is even impossible when the sample-of-interest co-exists with each other in one collectable unit, e.g., multiple visual instances in one image. Therefore, long-tailed classification is the key to deep learning at scale. However, existing methods are mainly based on re-weighting/re-sampling heuristics that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution. Specifically, our theory shows that the SGD momentum is essentially a confounder in long-tailed classification. On one hand, it has a harmful causal effect that misleads the tail prediction biased towards the head. On the other hand, its induced mediation also benefits the representation learning and head prediction. Our framework elegantly disentangles the paradoxical effects of the momentum, by pursuing the direct causal effect caused by an input sample. In particular, we use causal intervention in training, and counterfactual reasoning in inference, to remove the "bad" while keep the "good". We achieve new state-of-the-arts on three long-tailed visual recognition benchmarks: Long-tailed CIFAR-10/-100, ImageNet-LT for image classification and LVIS for instance segmentation.
This paper is accepted by NeurIPS 2020. The code is available on GitHub: https://github.com/KaihuaTang/Long-Tailed-Recognition.pytorch
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- SMOTE: Synthetic Minority Over-sampling Technique
- BPR: Bayesian Personalized Ranking from Implicit Feedback
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Learning to Reweight Examples for Robust Deep Learning
- MMDetection: Open MMLab Detection Toolbox and Benchmark
- Interventional Few-Shot Learning
- Large-Scale Long-Tailed Recognition in an Open World
- Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective
- Equalization Loss for Long-Tailed Object Recognition
- Counterfactual VQA: A Cause-Effect Look at Language Bias
- Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
Cited by in corpus (4)
- A Survey on Long-Tailed Visual Recognition
- Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection
- Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features
- Counterfactual Variable Control for Robust and Interpretable Question Answering