Large-Scale Long-Tailed Recognition in an Open World

Abstract

Real world data often have a long-tailed and open-ended distribution. A practical recognition system must classify among majority and minority classes, generalize from a few known instances, and acknowledge novelty upon a never seen instance. We define Open Long-Tailed Recognition (OLTR) as learning from such naturally distributed data and optimizing the classification accuracy over a balanced test set which include head, tail, and open classes. OLTR must handle imbalanced classification, few-shot learning, and open-set recognition in one integrated algorithm, whereas existing classification approaches focus only on one aspect and deliver poorly over the entire class spectrum. The key challenges are how to share visual knowledge between head and tail classes and how to reduce confusion between tail and open classes. We develop an integrated OLTR algorithm that maps an image to a feature space such that visual concepts can easily relate to each other based on a learned metric that respects the closed-world classification while acknowledging the novelty of the open world. Our so-called dynamic meta-embedding combines a direct image feature and an associated memory feature, with the feature norm indicating the familiarity to known classes. On three large-scale OLTR datasets we curate from object-centric ImageNet, scene-centric Places, and face-centric MS1M data, our method consistently outperforms the state-of-the-art. Our code, datasets, and models enable future OLTR research and are publicly available at https://liuziwei7.github.io/projects/LongTail.html.

To appear in CVPR 2019 as an oral presentation. Code, datasets and models are available at https://liuziwei7.github.io/projects/LongTail.html

References in corpus (13)

• Prototypical Networks for Few-shot Learning
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• Episodic Curiosity through Reachability
• Learning Adversarially Fair and Transferable Representations
• Using Fast Weights to Attend to the Recent Past
• Non-local Neural Networks
• The Devil is in the Tails: Fine-grained Classification in the Wild
• Learning to cluster in order to transfer across domains and tasks
• Women also Snowboard: Overcoming Bias in Captioning Models
• Low-Shot Learning from Imaginary Data
• Incremental Few-Shot Learning with Attention Attractor Networks

Cited by in corpus (21)

• Recent Advances in Open Set Recognition: A Survey
• Delving into Deep Imbalanced Regression
• BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition
• Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect
• CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning
• A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries
• MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
• Data-Efficient Pretraining via Contrastive Self-Supervision
• Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details
• Generalized Zero-shot ICD Coding
• Few-Shot Open-Set Recognition using Meta-Learning
• Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection
• Distributional Robustness Loss for Long-tail Learning
• Exemplar-Based Open-Set Panoptic Segmentation Network
• Recovering the Unbiased Scene Graphs from the Biased Ones
• Energy Aligning for Biased Models
• Offense Detection in Dravidian Languages using Code-Mixing Index based Focal Loss
• Instance-Conditioned GAN
• On-target Adaptation
• A Linkage-based Doubly Imbalanced Graph Learning Framework for Face Clustering