Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples

Abstract

In the field of intelligent multimedia analysis, ultra-fine-grained visual categorization (Ultra-FGVC) plays a vital role in distinguishing intricate subcategories within broader categories. However, this task is inherently challenging due to the complex granularity of category subdivisions and the limited availability of data for each category. To address these challenges, this work proposes CSDNet, a pioneering framework that effectively explores contrastive learning and self-distillation to learn discriminative representations specifically designed for Ultra-FGVC tasks. CSDNet comprises three main modules: Subcategory-Specific Discrepancy Parsing (SSDP), Dynamic Discrepancy Learning (DDL), and Subcategory-Specific Discrepancy Transfer (SSDT), which collectively enhance the generalization of deep models across instance, feature, and logit prediction levels. To increase the diversity of training samples, the SSDP module introduces adaptive augmented samples to spotlight subcategory-specific discrepancies. Simultaneously, the proposed DDL module stores historical intermediate features by a dynamic memory queue, which optimizes the feature learning space through iterative contrastive learning. Furthermore, the SSDT module effectively distills subcategory-specific discrepancies knowledge from the inherent structure of limited training data using a self-distillation paradigm at the logit prediction level. Experimental results demonstrate that CSDNet outperforms current state-of-the-art Ultra-FGVC methods, emphasizing its powerful efficacy and adaptability in addressing Ultra-FGVC tasks.

Accepted for Publication in TCSVT

References in corpus (12)

• Distilling the Knowledge in a Neural Network
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
• Improved Regularization of Convolutional Neural Networks with Cutout
• Improved Baselines with Momentum Contrastive Learning
• Diversified Visual Attention Networks for Fine-Grained Object Classification
• Weakly Supervised Attention Pyramid Convolutional Neural Network for Fine-Grained Visual Classification
• Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment
• Pay attention to the activations: a modular attention mechanism for fine-grained image recognition
• Feature Fusion Vision Transformer for Fine-Grained Visual Categorization
• A Continual Development Methodology for Large-scale Multitask Dynamic ML Systems
• ViT-FOD: A Vision Transformer based Fine-grained Object Discriminator

Cited by in corpus (1)

• IFShip: Interpretable Fine-grained Ship Classification with Domain Knowledge-Enhanced Vision-Language Models