A Multi-Stage Multi-Task Neural Network for Aerial Scene Interpretation and Geolocalization

Abstract

Semantic segmentation and vision-based geolocalization in aerial images are challenging tasks in computer vision. Due to the advent of deep convolutional nets and the availability of relatively low cost UAVs, they are currently generating a growing attention in the field. We propose a novel multi-task multi-stage neural network that is able to handle the two problems at the same time, in a single forward pass. The first stage of our network predicts pixelwise class labels, while the second stage provides a precise location using two branches. One branch uses a regression network, while the other is used to predict a location map trained as a segmentation task. From a structural point of view, our architecture uses encoder-decoder modules at each stage, having the same encoder structure re-used. Furthermore, its size is limited to be tractable on an embedded GPU. We achieve commercial GPS-level localization accuracy from satellite images with spatial resolution of 1 square meter per pixel in a city-wide area of interest. On the task of semantic segmentation, we obtain state-of-the-art results on two challenging datasets, the Inria Aerial Image Labeling dataset and Massachusetts Buildings.

23 pages, 11 figures. Under review at the 15th European Conference on Computer Vision (ECCV 2018)

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Overview of Multi-Task Learning in Deep Neural Networks
• Densely Connected Convolutional Networks
• A Review on Deep Learning Techniques Applied to Semantic Segmentation
• PlaNet - Photo Geolocation with Convolutional Neural Networks
• Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery
• Pixels to Graphs by Associative Embedding
• The Importance of Skip Connections in Biomedical Image Segmentation
• PixelNet: Representation of the pixels, by the pixels, and for the pixels
• Effective Use of Dilated Convolutions for Segmenting Small Object Instances in Remote Sensing Imagery

Cited by in corpus (2)

• Learning Navigation by Visual Localization and Trajectory Prediction
• Contextual Pyramid Attention Network for Building Segmentation in Aerial Imagery