The Devil is in the Tails: Fine-grained Classification in the Wild

Abstract

The world is long-tailed. What does this mean for computer vision and visual recognition? The main two implications are (1) the number of categories we need to consider in applications can be very large, and (2) the number of training examples for most categories can be very small. Current visual recognition algorithms have achieved excellent classification accuracy. However, they require many training examples to reach peak performance, which suggests that long-tailed distributions will not be dealt with well. We analyze this question in the context of eBird, a large fine-grained classification dataset, and a state-of-the-art deep network classification algorithm. We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods. Our findings suggest that our community should come to grips with the question of long tails.

References in corpus (2)

• What makes ImageNet good for transfer learning?
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition

Cited by in corpus (23)

• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• Class-Balanced Loss Based on Effective Number of Samples
• What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
• Large-Scale Long-Tailed Recognition in an Open World
• A Study of the Generalizability of Self-Supervised Representations
• Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia
• Disentangling Label Distribution for Long-tailed Visual Recognition
• Balanced Knowledge Distillation for Long-tailed Learning
• Imbalanced Continual Learning with Partitioning Reservoir Sampling
• Fine-Grained Visual Classification of Plant Species In The Wild: Object Detection as A Reinforced Means of Attention
• Generalized Data Weighting via Class-level Gradient Manipulation
• Teacher's pet: understanding and mitigating biases in distillation
• A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
• MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
• A surrogate loss function for optimization of $F_{\beta }$ score in binary classification with imbalanced data
• Copula-Based Normalizing Flows
• Image-to-Image Translation of Synthetic Samples for Rare Classes
• Analysis of Video Feature Learning in Two-Stream CNNs on the Example of Zebrafish Swim Bout Classification
• Beyond image classification: zooplankton identification with deep vector space embeddings
• Domain Adaptation for Rare Classes Augmented with Synthetic Samples
• When in Doubt, Summon the Titans: Efficient Inference with Large Models
• Metadata Shaping: Natural Language Annotations for the Tail
• Training Over-parameterized Models with Non-decomposable Objectives