SpaceNet MVOI: a Multi-View Overhead Imagery Dataset

Abstract

Detection and segmentation of objects in overheard imagery is a challenging task. The variable density, random orientation, small size, and instance-to-instance heterogeneity of objects in overhead imagery calls for approaches distinct from existing models designed for natural scene datasets. Though new overhead imagery datasets are being developed, they almost universally comprise a single view taken from directly overhead ("at nadir"), failing to address a critical variable: look angle. By contrast, views vary in real-world overhead imagery, particularly in dynamic scenarios such as natural disasters where first looks are often over 40 degrees off-nadir. This represents an important challenge to computer vision methods, as changing view angle adds distortions, alters resolution, and changes lighting. At present, the impact of these perturbations for algorithmic detection and segmentation of objects is untested. To address this problem, we present an open source Multi-View Overhead Imagery dataset, termed SpaceNet MVOI, with 27 unique looks from a broad range of viewing angles (-32.5 degrees to 54.0 degrees). Each of these images cover the same 665 square km geographic extent and are annotated with 126,747 building footprint labels, enabling direct assessment of the impact of viewpoint perturbation on model performance. We benchmark multiple leading segmentation and object detection models on: (1) building detection, (2) generalization to unseen viewing angles and resolutions, and (3) sensitivity of building footprint extraction to changes in resolution. We find that state of the art segmentation and object detection models struggle to identify buildings in off-nadir imagery and generalize poorly to unseen views, presenting an important benchmark to explore the broadly relevant challenge of detecting small, heterogeneous target objects in visually dynamic contexts.

arXiv commentAccepted into IEEE International Conference on Computer Vision (ICCV) 2019

References in corpus (10)

• MOT16: A Benchmark for Multi-Object Tracking
• DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images
• TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation
• SpaceNet: A Remote Sensing Dataset and Challenge Series
• Spatially Transformed Adversarial Examples
• You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery
• Why do deep convolutional networks generalize so poorly to small image transformations?
• A Large Dataset of Object Scans
• xView: Objects in Context in Overhead Imagery
• Unsupervised Learning of Visual Structure using Predictive Generative Networks

Cited by in corpus (13)

• Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges
• S2Looking: A Satellite Side-Looking Dataset for Building Change Detection
• iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images
• SpaceNet 6: Multi-Sensor All Weather Mapping Dataset
• Road Network and Travel Time Extraction from Multiple Look Angles with SpaceNet Data
• Accelerating Deep Learning Applications in Space
• RarePlanes: Synthetic Data Takes Flight
• The SpaceNet Multi-Temporal Urban Development Challenge
• The MIS Check-Dam Dataset for Object Detection and Instance Segmentation Tasks
• Gaussian Building Mesh (GBM): Extract a Building's 3D Mesh with Google Earth and Gaussian Splatting
• Improving Building Segmentation for Off-Nadir Satellite Imagery
• Learning Geocentric Object Pose in Oblique Monocular Images
• LUAI Challenge 2021 on Learning to Understand Aerial Images