Object-Part Attention Model for Fine-grained Image Classification

Abstract

Fine-grained image classification is to recognize hundreds of subcategories belonging to the same basic-level category, such as 200 subcategories belonging to the bird, which is highly challenging due to large variance in the same subcategory and small variance among different subcategories. Existing methods generally first locate the objects or parts and then discriminate which subcategory the image belongs to. However, they mainly have two limitations: (1) Relying on object or part annotations which are heavily labor consuming. (2) Ignoring the spatial relationships between the object and its parts as well as among these parts, both of which are significantly helpful for finding discriminative parts. Therefore, this paper proposes the object-part attention model (OPAM) for weakly supervised fine-grained image classification, and the main novelties are: (1) Object-part attention model integrates two level attentions: object-level attention localizes objects of images, and part-level attention selects discriminative parts of object. Both are jointly employed to learn multi-view and multi-scale features to enhance their mutual promotions. (2) Object-part spatial constraint model combines two spatial constraints: object spatial constraint ensures selected parts highly representative, and part spatial constraint eliminates redundancy and enhances discrimination of selected parts. Both are jointly employed to exploit the subtle and local differences for distinguishing the subcategories. Importantly, neither object nor part annotations are used in our proposed approach, which avoids the heavy labor consumption of labeling. Comparing with more than 10 state-of-the-art methods on 4 widely-used datasets, our OPAM approach achieves the best performance.

14 pages, submitted to IEEE Transactions on Image Processing

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Bird Species Categorization Using Pose Normalized Deep Convolutional Nets
• Diversified Visual Attention Networks for Fine-Grained Object Classification
• Attention for Fine-Grained Categorization
• A comparison of dense region detectors for image search and fine-grained classification
• Generalized Max Pooling
• InterActive: Inter-Layer Activeness Propagation
• Learning a Discriminative Filter Bank within a CNN for Fine-grained Recognition

Cited by in corpus (41)

• Attention Mechanisms in Computer Vision: A Survey
• The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification
• GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
• GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
• Delving Deep into Label Smoothing
• Low-Rank Pairwise Alignment Bilinear Network For Few-Shot Fine-Grained Image Classification
• SR-GNN: Spatial Relation-aware Graph Neural Network for Fine-Grained Image Categorization
• Transformer with Peak Suppression and Knowledge Guidance for Fine-grained Image Recognition
• Pay attention to the activations: a modular attention mechanism for fine-grained image recognition
• Part-guided Relational Transformers for Fine-grained Visual Recognition
• SAC-Net: Spatial Attenuation Context for Salient Object Detection
• Sequential vessel segmentation via deep channel attention network
• Relational Knowledge Distillation
• Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning
• Object Discovery From a Single Unlabeled Image by Mining Frequent Itemset With Multi-scale Features
• Part-Aware Fine-grained Object Categorization using Weakly Supervised Part Detection Network
• Deep learning for multi-label classification of coral conditions in the Indo-Pacific via underwater photogrammetry
• Mask Guided Attention For Fine-Grained Patchy Image Classification
• Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring
• Instance-Level Microtubule Tracking
• Look-into-Object: Self-supervised Structure Modeling for Object Recognition
• Fine-Grained Image Analysis with Deep Learning: A Survey
• Large-Scale Classification of Structured Objects using a CRF with Deep Class Embedding
• Fine-Grained Image Captioning with Global-Local Discriminative Objective
• Unsupervised Part Mining for Fine-grained Image Classification
• Fine-Grained Visual Classification of Plant Species In The Wild: Object Detection as A Reinforced Means of Attention
• Parsing Objects at a Finer Granularity: A Survey
• M2Former: Multi-Scale Patch Selection for Fine-Grained Visual Recognition
• IAUnet: Global Context-Aware Feature Learning for Person Re-Identification
• Your "Flamingo" is My "Bird": Fine-Grained, or Not
• Data-driven Meta-set Based Fine-Grained Visual Classification
• Group Based Deep Shared Feature Learning for Fine-grained Image Classification
• Attribute-Aware Attention Model for Fine-grained Representation Learning
• Sequential Random Network for Fine-grained Image Classification
• Learning a Discriminant Latent Space with Neural Discriminant Analysis
• Fine-Grained Visual Classification via Simultaneously Learning of Multi-regional Multi-grained Features
• What I See Is What You See: Joint Attention Learning for First and Third Person Video Co-analysis
• Distance Metric Learned Collaborative Representation Classifier
• Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning
• Competing Ratio Loss for Discriminative Multi-class Image Classification
• Interpretable Attention Guided Network for Fine-grained Visual Classification