Towards seamless multi-view scene analysis from satellite to street-level

Abstract

In this paper, we discuss and review how combined multi-view imagery from satellite to street-level can benefit scene analysis. Numerous works exist that merge information from remote sensing and images acquired from the ground for tasks like land cover mapping, object detection, or scene understanding. What makes the combination of overhead and street-level images challenging, is the strongly varying viewpoint, different scale, illumination, sensor modality and time of acquisition. Direct (dense) matching of images on a per-pixel basis is thus often impossible, and one has to resort to alternative strategies that will be discussed in this paper. We review recent works that attempt to combine images taken from the ground and overhead views for purposes like scene registration, reconstruction, or classification. Three methods that represent the wide range of potential methods and applications (change detection, image orientation, and tree cataloging) are described in detail. We show that cross-fertilization between remote sensing, computer vision and machine learning is very valuable to make the best of geographic data available from Earth Observation sensors and ground imagery. Despite its challenges, we believe that integrating these complementary data sources will lead to major breakthroughs in Big GeoData.

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Recent Advances in Domain Adaptation for the Classification of Remote Sensing Data
• Localizing and Orienting Street Views Using Overhead Imagery
• Wide-Area Image Geolocalization with Aerial Reference Imagery

Cited by in corpus (13)

• Deep learning in remote sensing: a review
• Understanding urban landuse from the above and ground perspectives: a deep learning, multimodal solution
• Computational Socioeconomics
• Automatic Discovery and Geotagging of Objects from Street View Imagery
• Towards a Collective Agenda on AI for Earth Science Data Analysis
• Simultaneous multi-view instance detection with learned geometric soft-constraints
• Change Detection between Multimodal Remote Sensing Data Using Siamese CNN
• Multisource and Multitemporal Data Fusion in Remote Sensing
• Bounding Boxes Are All We Need: Street View Image Classification via Context Encoding of Detected Buildings
• Change detection needs change information: improving deep 3D point cloud change detection
• GeoGraph: Learning graph-based multi-view object detection with geometric cues end-to-end
• Urban land-use analysis using proximate sensing imagery: a survey
• Image-to-GPS Verification Through A Bottom-Up Pattern Matching Network