Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification

Abstract

Aerial image classification is of great significance in remote sensing community, and many researches have been conducted over the past few years. Among these studies, most of them focus on categorizing an image into one semantic label, while in the real world, an aerial image is often associated with multiple labels, e.g., multiple object-level labels in our case. Besides, a comprehensive picture of present objects in a given high resolution aerial image can provide more in-depth understanding of the studied region. For these reasons, aerial image multi-label classification has been attracting increasing attention. However, one common limitation shared by existing methods in the community is that the co-occurrence relationship of various classes, so called class dependency, is underexplored and leads to an inconsiderate decision. In this paper, we propose a novel end-to-end network, namely class-wise attention-based convolutional and bidirectional LSTM network (CA-Conv-BiLSTM), for this task. The proposed network consists of three indispensable components: 1) a feature extraction module, 2) a class attention learning layer, and 3) a bidirectional LSTM-based sub-network. Particularly, the feature extraction module is designed for extracting fine-grained semantic feature maps, while the class attention learning layer aims at capturing discriminative class-specific features. As the most important part, the bidirectional LSTM-based sub-network models the underlying class dependency in both directions and produce structured multiple object labels. Experimental results on UCM multi-label dataset and DFC15 multi-label dataset validate the effectiveness of our model quantitatively and qualitatively.

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Deep learning in remote sensing: a review
• Fully Convolutional Networks for Semantic Segmentation
• Remote Sensing Image Scene Classification: Benchmark and State of the Art
• AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification
• Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Building Instance Classification Using Street View Images
• Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models
• Vehicle Instance Segmentation from Aerial Image and Video Using a Multi-Task Learning Residual Fully Convolutional Network
• A Unified View of Multi-Label Performance Measures
• IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network
• RiFCN: Recurrent Network in Fully Convolutional Network for Semantic Segmentation of High Resolution Remote Sensing Images

Cited by in corpus (16)

• Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Relation Network for Multi-label Aerial Image Classification
• Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification
• A framework for large-scale mapping of human settlement extent from Sentinel-2 images via fully convolutional neural networks
• Pancreas segmentation with probabilistic map guided bi-directional recurrent UNet
• Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classification
• A Novel Multi-Attention Driven System For Multi-Label Remote Sensing Image Classification
• Multi-label Image Classification using Adaptive Graph Convolutional Networks: from a Single Domain to Multiple Domains
• Weakly Supervised Instance Attention for Multisource Fine-Grained Object Recognition with an Application to Tree Species Classification
• MultiScene: A Large-scale Dataset and Benchmark for Multi-scene Recognition in Single Aerial Images
• BigEarthNet Dataset with A New Class-Nomenclature for Remote Sensing Image Understanding
• A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes
• A hierarchical deep learning framework for the consistent classification of land use objects in geospatial databases
• Adaptive Gradient Calibration for Single-Positive Multi-Label Learning in Remote Sensing Image Scene Classification
• Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks