BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition

Abstract

Our work focuses on tackling the challenging but natural visual recognition task of long-tailed data distribution (i.e., a few classes occupy most of the data, while most classes have rarely few samples). In the literature, class re-balancing strategies (e.g., re-weighting and re-sampling) are the prominent and effective methods proposed to alleviate the extreme imbalance for dealing with long-tailed problems. In this paper, we firstly discover that these re-balancing methods achieving satisfactory recognition accuracy owe to that they could significantly promote the classifier learning of deep networks. However, at the same time, they will unexpectedly damage the representative ability of the learned deep features to some extent. Therefore, we propose a unified Bilateral-Branch Network (BBN) to take care of both representation learning and classifier learning simultaneously, where each branch does perform its own duty separately. In particular, our BBN model is further equipped with a novel cumulative learning strategy, which is designed to first learn the universal patterns and then pay attention to the tail data gradually. Extensive experiments on four benchmark datasets, including the large-scale iNaturalist ones, justify that the proposed BBN can significantly outperform state-of-the-art methods. Furthermore, validation experiments can demonstrate both our preliminary discovery and effectiveness of tailored designs in BBN for long-tailed problems. Our method won the first place in the iNaturalist 2019 large scale species classification competition, and our code is open-source and available at https://github.com/Megvii-Nanjing/BBN.

arXiv commentAccepted by CVPR 2020; Our work won the first place in the iNaturalist 2019 large scale species classification competition, and our code is open-source and available at https://github.com/Megvii-Nanjing/BBN

References in corpus (14)

• mixup: Beyond Empirical Risk Minimization
• A systematic study of the class imbalance problem in convolutional neural networks
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Focal Loss for Dense Object Detection
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Survey of resampling techniques for improving classification performance in unbalanced datasets
• The Devil is in the Tails: Fine-grained Classification in the Wild
• Class-Balanced Loss Based on Effective Number of Samples
• What is the Effect of Importance Weighting in Deep Learning?
• RPC: A Large-Scale Retail Product Checkout Dataset
• One-shot Face Recognition by Promoting Underrepresented Classes
• Deep Learning for Fine-Grained Image Analysis: A Survey
• Large-Scale Long-Tailed Recognition in an Open World
• Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

Cited by in corpus (12)

• Balanced Meta-Softmax for Long-Tailed Visual Recognition
• Rethinking the Value of Labels for Improving Class-Imbalanced Learning
• Feature Fusion from Head to Tail for Long-Tailed Visual Recognition
• Rethinking of Pedestrian Attribute Recognition: Realistic Datasets with Efficient Method
• Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia
• MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition
• ELF: An Early-Exiting Framework for Long-Tailed Classification
• The Devil is the Classifier: Investigating Long Tail Relation Classification with Decoupling Analysis
• GNN-XML: Graph Neural Networks for Extreme Multi-label Text Classification
• Meta Feature Modulator for Long-tailed Recognition
• Posterior Re-calibration for Imbalanced Datasets
• Parametric Contrastive Learning