Dense semantic labeling of sub-decimeter resolution images with convolutional neural networks

Abstract

Semantic labeling (or pixel-level land-cover classification) in ultra-high resolution imagery (< 10cm) requires statistical models able to learn high level concepts from spatial data, with large appearance variations. Convolutional Neural Networks (CNNs) achieve this goal by learning discriminatively a hierarchy of representations of increasing abstraction. In this paper we present a CNN-based system relying on an downsample-then-upsample architecture. Specifically, it first learns a rough spatial map of high-level representations by means of convolutions and then learns to upsample them back to the original resolution by deconvolutions. By doing so, the CNN learns to densely label every pixel at the original resolution of the image. This results in many advantages, including i) state-of-the-art numerical accuracy, ii) improved geometric accuracy of predictions and iii) high efficiency at inference time. We test the proposed system on the Vaihingen and Potsdam sub-decimeter resolution datasets, involving semantic labeling of aerial images of 9cm and 5cm resolution, respectively. These datasets are composed by many large and fully annotated tiles allowing an unbiased evaluation of models making use of spatial information. We do so by comparing two standard CNN architectures to the proposed one: standard patch classification, prediction of local label patches by employing only convolutions and full patch labeling by employing deconvolutions. All the systems compare favorably or outperform a state-of-the-art baseline relying on superpixels and powerful appearance descriptors. The proposed full patch labeling CNN outperforms these models by a large margin, also showing a very appealing inference time.

arXiv commentAccepted in IEEE Transactions on Geoscience and Remote Sensing, 2016

References in corpus (2)

• Unsupervised Deep Feature Extraction for Remote Sensing Image Classification
• Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions

Cited by in corpus (35)

• Deep learning in remote sensing: a review
• ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
• Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Models
• Deep Learning for Classification of Hyperspectral Data: A Comparative Review
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Self-attention for raw optical Satellite Time Series Classification
• Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network
• RSVQA: Visual Question Answering for Remote Sensing Data
• Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models
• High-Resolution Semantic Labeling with Convolutional Neural Networks
• EddyNet: A Deep Neural Network For Pixel-Wise Classification of Oceanic Eddies
• Vehicle Instance Segmentation from Aerial Image and Video Using a Multi-Task Learning Residual Fully Convolutional Network
• Understanding urban landuse from the above and ground perspectives: a deep learning, multimodal solution
• Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Images
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• Aerial Imagery for Roof Segmentation: A Large-Scale Dataset towards Automatic Mapping of Buildings
• Dynamic Multi-Context Segmentation of Remote Sensing Images based on Convolutional Networks
• Polarimetric Convolutional Network for PolSAR Image Classification
• Deep multi-task learning for a geographically-regularized semantic segmentation of aerial images
• A framework for large-scale mapping of human settlement extent from Sentinel-2 images via fully convolutional neural networks
• Recurrent Neural Networks to Correct Satellite Image Classification Maps
• Correcting rural building annotations in OpenStreetMap using convolutional neural networks
• Urban Land Cover Classification with Missing Data Modalities Using Deep Convolutional Neural Networks
• Recurrent Multiresolution Convolutional Networks for VHR Image Classification
• Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data
• A framework for remote sensing images processing using deep learning technique
• Decision fusion with multiple spatial supports by conditional random fields
• Semantic Labeling of High Resolution Images Using EfficientUNets and Transformers
• A Comparison and Strategy of Semantic Segmentation on Remote Sensing Images
• Deploying machine learning to assist digital humanitarians: making image annotation in OpenStreetMap more efficient
• LULC classification by semantic segmentation of satellite images using FastFCN
• A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagery
• Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation
• IRSAMap:Towards Large-Scale, High-Resolution Land Cover Map Vectorization
• Learning with less: label-efficient land cover classification at very high spatial resolution using self-supervised deep learning