Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection

Abstract

Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and detection. To deal with this challenge, image resampling is typically introduced as a simple but effective approach. However, we observe that long-tailed detection differs from classification since multiple classes may be present in one image. As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object level. We address object-level resampling by introducing an object-centric memory replay strategy based on dynamic, episodic memory banks. Our proposed strategy has two benefits: 1) convenient object-level resampling without significant extra computation, and 2) implicit feature-level augmentation from model updates. We show that image-level and object-level resamplings are both important, and thus unify them with a joint resampling strategy (RIO). Our method outperforms state-of-the-art long-tailed detection and segmentation methods on LVIS v0.5 across various backbones. Code is available at https://github.com/NVlabs/RIO.

arXiv commentAccepted to ICML 2021

References in corpus (12)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Improved Baselines with Momentum Contrastive Learning
• Frustratingly Simple Few-Shot Object Detection
• Decoupling Representation and Classifier for Long-Tailed Recognition
• A Closer Look at Few-shot Classification
• Low-Shot Learning from Imaginary Data
• Large-Scale Long-Tailed Recognition in an Open World
• Equalization Loss for Long-Tailed Object Recognition
• Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
• Seesaw Loss for Long-Tailed Instance Segmentation