Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations

Abstract

Self-supervision based deep learning classification approaches have received considerable attention in academic literature. However, the performance of such methods on remote sensing imagery domains remains under-explored. In this work, we explore contrastive representation learning methods on the task of imagery-based city classification, an important problem in urban computing. We use satellite and map imagery across 2 domains, 3 million locations and more than 1500 cities. We show that self-supervised methods can build a generalizable representation from as few as 200 cities, with representations achieving over 95\% accuracy in unseen cities with minimal additional training. We also find that the performance discrepancy of such methods, when compared to supervised methods, induced by the domain discrepancy between natural imagery and abstract imagery is significant for remote sensing imagery. We compare all analysis against existing supervised models from academic literature and open-source our models for broader usage and further criticism.

arXiv commentPreprint of https://ieeexplore.ieee.org/document/9647061

References in corpus (13)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Bootstrap your own latent: A new approach to self-supervised Learning
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Unsupervised Data Augmentation for Consistency Training
• Unsupervised Representation Learning by Predicting Image Rotations
• Large Scale Adversarial Representation Learning
• Identifying safe intersection design through unsupervised feature extraction from satellite imagery
• Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
• The 'Paris-end' of town? Urban typology through machine learning
• The Nature of Human Settlement: Building an understanding of high performance city design

Cited by in corpus (3)

• Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
• Self-Supervised Vision Transformers for Malware Detection
• DALLE-URBAN: Capturing the urban design expertise of large text to image transformers