Multi-Granularity Canonical Appearance Pooling for Remote Sensing Scene Classification

Abstract

Recognising remote sensing scene images remains challenging due to large visual-semantic discrepancies. These mainly arise due to the lack of detailed annotations that can be employed to align pixel-level representations with high-level semantic labels. As the tagging process is labour-intensive and subjective, we hereby propose a novel Multi-Granularity Canonical Appearance Pooling (MG-CAP) to automatically capture the latent ontological structure of remote sensing datasets. We design a granular framework that allows progressively cropping the input image to learn multi-grained features. For each specific granularity, we discover the canonical appearance from a set of pre-defined transformations and learn the corresponding CNN features through a maxout-based Siamese style architecture. Then, we replace the standard CNN features with Gaussian covariance matrices and adopt the proper matrix normalisations for improving the discriminative power of features. Besides, we provide a stable solution for training the eigenvalue-decomposition function (EIG) in a GPU and demonstrate the corresponding back-propagation using matrix calculus. Extensive experiments have shown that our framework can achieve promising results in public remote sensing scene datasets.

This paper is going to be published by IEEE Transactions on Image Processing

References in corpus (6)

• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Remote Sensing Image Scene Classification: Benchmark and State of the Art
• AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification
• A Survey on Object Detection in Optical Remote Sensing Images
• Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification
• Locality and Structure Regularized Low Rank Representation for Hyperspectral Image Classification

Cited by in corpus (8)

• An Empirical Study of Remote Sensing Pretraining
• Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion
• MGML: Multi-Granularity Multi-Level Feature Ensemble Network for Remote Sensing Scene Classification
• All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification
• Enhanced Multi-level Features for Very High Resolution Remote Sensing Scene Classification
• AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Framework
• Dual Contrastive Network for Few-Shot Remote Sensing Image Scene Classification
• Invariant Deep Compressible Covariance Pooling for Aerial Scene Categorization