All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification

Abstract

Aerial scene classification remains challenging as: 1) the size of key objects in determining the scene scheme varies greatly; 2) many objects irrelevant to the scene scheme are often flooded in the image. Hence, how to effectively perceive the region of interests (RoIs) from a variety of sizes and build more discriminative representation from such complicated object distribution is vital to understand an aerial scene. In this paper, we propose a novel all grains, one scheme (AGOS) framework to tackle these challenges. To the best of our knowledge, it is the first work to extend the classic multiple instance learning into multi-grain formulation. Specially, it consists of a multi-grain perception module (MGP), a multi-branch multi-instance representation module (MBMIR) and a self-aligned semantic fusion (SSF) module. Firstly, our MGP preserves the differential dilated convolutional features from the backbone, which magnifies the discriminative information from multi-grains. Then, our MBMIR highlights the key instances in the multi-grain representation under the MIL formulation. Finally, our SSF allows our framework to learn the same scene scheme from multi-grain instance representations and fuses them, so that the entire framework is optimized as a whole. Notably, our AGOS is flexible and can be easily adapted to existing CNNs in a plug-and-play manner. Extensive experiments on UCM, AID and NWPU benchmarks demonstrate that our AGOS achieves a comparable performance against the state-of-the-art methods.

arXiv comment16 pages, 9 figures

References in corpus (23)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep learning in remote sensing: a review
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Remote Sensing Image Scene Classification: Benchmark and State of the Art
• AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification
• Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Models
• Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities
• Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification
• Attention-based Deep Multiple Instance Learning
• Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges
• Revisiting Multiple Instance Neural Networks
• Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data
• MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification
• Vehicle Instance Segmentation from Aerial Image and Video Using a Multi-Task Learning Residual Fully Convolutional Network
• Multiple instance dense connected convolution neural network for aerial image scene classification
• Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
• Multi-Granularity Canonical Appearance Pooling for Remote Sensing Scene Classification
• Learning RoI Transformer for Detecting Oriented Objects in Aerial Images
• Relation Network for Multi-label Aerial Image Classification
• Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
• Multi-Instance Learning with Any Hypothesis Class
• Learning and Interpreting Multi-Multi-Instance Learning Networks