The Devil is in the Tails: Fine-grained Classification in the Wild

Abstract

The world is long-tailed. What does this mean for computer vision and visual recognition? The main two implications are (1) the number of categories we need to consider in applications can be very large, and (2) the number of training examples for most categories can be very small. Current visual recognition algorithms have achieved excellent classification accuracy. However, they require many training examples to reach peak performance, which suggests that long-tailed distributions will not be dealt with well. We analyze this question in the context of eBird, a large fine-grained classification dataset, and a state-of-the-art deep network classification algorithm. We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods. Our findings suggest that our community should come to grips with the question of long tails.

References in corpus (7)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Fine-Grained Visual Classification of Aircraft
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• What makes ImageNet good for transfer learning?
• Neural Activation Constellations: Unsupervised Part Model Discovery with Convolutional Networks
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• Metric Learning with Adaptive Density Discrimination

Cited by in corpus (47)

• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• Class-Balanced Loss Based on Effective Number of Samples
• What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
• BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition
• Large-Scale Long-Tailed Recognition in an Open World
• Remix: Rebalanced Mixup
• Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning
• Long-tail learning via logit adjustment
• Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets
• A Study of the Generalizability of Self-Supervised Representations
• Recognition in Terra Incognita
• Feature Transfer Learning for Deep Face Recognition with Under-Represented Data
• Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia
• Disentangling Label Distribution for Long-tailed Visual Recognition
• Learning Imbalanced Datasets with Maximum Margin Loss
• Balanced Knowledge Distillation for Long-tailed Learning
• Imbalanced Continual Learning with Partitioning Reservoir Sampling
• MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions
• Prototypical Clustering Networks for Dermatological Disease Diagnosis
• AutoScale: Learning to Scale for Crowd Counting and Localization
• Fine-Grained Visual Classification of Plant Species In The Wild: Object Detection as A Reinforced Means of Attention
• Geo-Aware Networks for Fine-Grained Recognition
• Object Segmentation of Cluttered Airborne LiDAR Point Clouds
• Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
• Generalized Data Weighting via Class-level Gradient Manipulation
• Teacher's pet: understanding and mitigating biases in distillation
• Classifier and Exemplar Synthesis for Zero-Shot Learning
• A surrogate loss function for optimization of $F_{\beta }$ score in binary classification with imbalanced data
• MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
• A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
• Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning
• CoCoNet: A Collaborative Convolutional Network
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces
• Trustworthy Long-Tailed Classification
• From Generalized zero-shot learning to long-tail with class descriptors
• Incremental Meta-Learning via Indirect Discriminant Alignment
• Copula-Based Normalizing Flows
• PProCRC: Probabilistic Collaboration of Image Patches
• Metadata Shaping: Natural Language Annotations for the Tail
• Analysis of Video Feature Learning in Two-Stream CNNs on the Example of Zebrafish Swim Bout Classification
• Beyond image classification: zooplankton identification with deep vector space embeddings
• Training Over-parameterized Models with Non-decomposable Objectives
• Image-to-Image Translation of Synthetic Samples for Rare Classes
• Domain Adaptation for Rare Classes Augmented with Synthetic Samples
• Procrustean Training for Imbalanced Deep Learning
• When in Doubt, Summon the Titans: Efficient Inference with Large Models