Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition

Abstract

Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches, which may not sufficiently facilitate meaningful object-specific semantic understanding, leading to a reliance on apparent background correlations. Moreover, they primarily rely on high-dimensional local descriptors to construct complex embedding space, potentially limiting the generalization. To address the above challenges, this article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition. RSaD introduces additional saliency-aware supervision via saliency detection to guide the model toward focusing on the intrinsic discriminative regions. Specifically, RSaD utilizes the saliency detection model to emphasize the critical regions of each sub-category, providing additional object-specific information for fine-grained prediction. RSaD transfers such information with two symmetric branches in a mutual learning paradigm. Furthermore, RSaD exploits inter-regional relationships to enhance the informativeness of the representation and subsequently summarize the highlighted details into contextual embeddings to facilitate the effective transfer, enabling quick generalization to novel sub-categories. The proposed approach is empirically evaluated on three widely used benchmarks, demonstrating its superior performance.

arXiv commentAccepted by IEEE Transactions on Multimedia

References in corpus (12)

• Distilling the Knowledge in a Neural Network
• U${}^2$-Net: Going Deeper with Nested U-Structure for Salient Object Detection
• The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification
• SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning
• BSNet: Bi-Similarity Network for Few-shot Fine-grained Image Classification
• Weakly Supervised Attention Pyramid Convolutional Neural Network for Fine-Grained Visual Classification
• Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications
• Low-Rank Pairwise Alignment Bilinear Network For Few-Shot Fine-Grained Image Classification
• Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples
• Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment
• TOAN: Target-Oriented Alignment Network for Fine-Grained Image Categorization with Few Labeled Samples
• Multi-level Second-order Few-shot Learning