Deep learning in remote sensing: a review

Abstract

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.

Accepted for publication IEEE Geoscience and Remote Sensing Magazine

References in corpus (15)

• Remote Sensing Image Scene Classification: Benchmark and State of the Art
• A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
• AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification
• A Survey on Object Detection in Optical Remote Sensing Images
• Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification
• Computing the Stereo Matching Cost with a Convolutional Neural Network
• Advances in Hyperspectral Image Classification: Earth monitoring with statistical learning methods
• Unsupervised Deep Feature Extraction for Remote Sensing Image Classification
• Recent Advances in Domain Adaptation for the Classification of Remote Sensing Data
• Dense semantic labeling of sub-decimeter resolution images with convolutional neural networks
• Learning Low Dimensional Convolutional Neural Networks for High-Resolution Remote Sensing Image Retrieval
• BASS Net: Band-Adaptive Spectral-Spatial Feature Learning Neural Network for Hyperspectral Image Classification
• Look Wider to Match Image Patches with Convolutional Neural Networks
• Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions
• Towards seamless multi-view scene analysis from satellite to street-level

Cited by in corpus (130)

• Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark
• ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data
• Deep Learning for Hyperspectral Image Classification: An Overview
• UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery
• DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images
• Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Models
• Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities
• Seeing biodiversity: perspectives in machine learning for wildlife conservation
• Deep Learning for Classification of Hyperspectral Data: A Comparative Review
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
• Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network
• Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation
• Classification of Hyperspectral and LiDAR Data Using Coupled CNNs
• A Review on Deep Learning in UAV Remote Sensing
• ABCNet: Attentive Bilateral Contextual Network for Efficient Semantic Segmentation of Fine-Resolution Remote Sensing Images
• Building Instance Classification Using Street View Images
• Detecting Mammals in UAV Images: Best Practices to address a substantially Imbalanced Dataset with Deep Learning
• Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors
• RSVQA: Visual Question Answering for Remote Sensing Data
• Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders
• MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery
• Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review
• Building extraction with vision transformer
• Country-wide high-resolution vegetation height mapping with Sentinel-2
• What, Where and How to Transfer in SAR Target Recognition Based on Deep CNNs
• Landslide4Sense: Reference Benchmark Data and Deep Learning Models for Landslide Detection
• Active Fire Detection in Landsat-8 Imagery: a Large-Scale Dataset and a Deep-Learning Study
• HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline
• Deep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection
• Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification
• Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, Applications, and Prospects
• EddyNet: A Deep Neural Network For Pixel-Wise Classification of Oceanic Eddies
• Vehicle Instance Segmentation from Aerial Image and Video Using a Multi-Task Learning Residual Fully Convolutional Network
• OpenStreetMap: Challenges and Opportunities in Machine Learning and Remote Sensing
• Understanding urban landuse from the above and ground perspectives: a deep learning, multimodal solution
• An Urban Water Extraction Method Combining Deep Learning and Google Earth Engine
• Hybrid Noise Removal in Hyperspectral Imagery With a Spatial-Spectral Gradient Network
• Multiple instance dense connected convolution neural network for aerial image scene classification
• Dynamic Multi-Context Segmentation of Remote Sensing Images based on Convolutional Networks
• InSAR Phase Denoising: A Review of Current Technologies and Future Directions
• R${}^3$-Net: A Deep Network for Multi-oriented Vehicle Detection in Aerial Images and Videos
• Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs
• Enabling Country-Scale Land Cover Mapping with Meter-Resolution Satellite Imagery
• Buildings Detection in VHR SAR Images Using Fully Convolution Neural Networks
• Mapping horizontal and vertical urban densification in Denmark with Landsat time-series from 1985 to 2018: a semantic segmentation solution
• Relation Network for Multi-label Aerial Image Classification
• Semantic Segmentation of Remote Sensing Images with Sparse Annotations
• Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification
• Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning
• Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
• CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding
• Self-Supervised Multisensor Change Detection
• Deep Learning for Satellite Image Time Series Analysis: A Review
• SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models
• From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data
• Towards a Collective Agenda on AI for Earth Science Data Analysis
• Predicting air quality via multimodal AI and satellite imagery
• A framework for large-scale mapping of human settlement extent from Sentinel-2 images via fully convolutional neural networks
• AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
• Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives
• Opening the Black-Box: A Systematic Review on Explainable AI in Remote Sensing
• Robust Self-Ensembling Network for Hyperspectral Image Classification
• Scattering Model Guided Adversarial Examples for SAR Target Recognition: Attack and Defense
• Building Footprint Generation by IntegratingConvolution Neural Network with Feature PairwiseConditional Random Field (FPCRF)
• Correcting rural building annotations in OpenStreetMap using convolutional neural networks
• Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes with Point-Level Annotations
• High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel-2 data with a deep learning approach
• Building Footprint Generation Using Improved Generative Adversarial Networks
• $\mathbf{\gamma }$-Net: Superresolving SAR Tomographic Inversion via Deep Learning
• Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data
• Semantic-aware Dense Representation Learning for Remote Sensing Image Change Detection
• All Grains, One Scheme (AGOS): Learning Multi-grain Instance Representation for Aerial Scene Classification
• Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives
• Unsupervised Deep Representation Learning and Few-Shot Classification of PolSAR Images
• There Are No Data Like More Data- Datasets for Deep Learning in Earth Observation
• Self-Supervised Learning for Invariant Representations from Multi-Spectral and SAR Images
• Self-supervised Audiovisual Representation Learning for Remote Sensing Data
• Decision fusion with multiple spatial supports by conditional random fields
• Emulation as an Accurate Alternative to Interpolation in Sampling Radiative Transfer Codes
• Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images
• Remote Sensing SpatioTemporal Vision-Language Models: A Comprehensive Survey
• Fusion of Heterogeneous Earth Observation Data for the Classification of Local Climate Zones
• Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training
• DML-GANR: Deep Metric Learning With Generative Adversarial Network Regularization for High Spatial Resolution Remote Sensing Image Retrieval
• MetaSegNet: Metadata-collaborative Vision-Language Representation Learning for Semantic Segmentation of Remote Sensing Images
• MUS-CDB: Mixed Uncertainty Sampling with Class Distribution Balancing for Active Annotation in Aerial Object Detection
• Fast Unmixing and Change Detection in Multitemporal Hyperspectral Data
• Building-road Collaborative Extraction from Remotely Sensed Images via Cross-Interaction
• A Biologically Interpretable Two-stage Deep Neural Network (BIT-DNN) For Vegetation Recognition From Hyperspectral Imagery
• Geomorphological Analysis Using Unpiloted Aircraft Systems, Structure from Motion, and Deep Learning
• Marine Snow Removal Benchmarking Dataset
• Deep Unsupervised Learning for 3D ALS Point Cloud Change Detection
• Semi-supervised Semantic Segmentation for Remote Sensing Images via Multi-scale Uncertainty Consistency and Cross-Teacher-Student Attention
• FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification
• Warping of Radar Data into Camera Image for Cross-Modal Supervision in Automotive Applications
• Road Network and Travel Time Extraction from Multiple Look Angles with SpaceNet Data
• Mapping Saturn using deep learning
• Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction from LiDAR Data with Limited Annotations
• AutoGeoLabel: Automated Label Generation for Geospatial Machine Learning
• SEnSeI: A Deep Learning Module for Creating Sensor Independent Cloud Masks
• Unsupervised Deep Features for Remote Sensing Image Matching via Discriminator Network
• Land Classification in Satellite Images by Injecting Traditional Features to CNN Models
• SenCLIP: Enhancing zero-shot land-use mapping for Sentinel-2 with ground-level prompting
• Building Footprint Extraction with Graph Convolutional Network
• Entropy Approximation by Machine Learning Regression: Application for Irregularity Evaluation of Images in Remote Sensing
• AdaTreeFormer: Few Shot Domain Adaptation for Tree Counting from a Single High-Resolution Image
• Automatic Extraction of Road Networks by using Teacher-Student Adaptive Structural Deep Belief Network and Its Application to Landslide Disaster
• Spatially explicit accuracy assessment of deep learning-based, fine-resolution built-up land data in the United States
• Remote Sensor Design for Visual Recognition with Convolutional Neural Networks
• A Tutorial on Modeling and Inference in Undirected Graphical Models for Hyperspectral Image Analysis
• Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning
• Recognition of polar lows in Sentinel-1 SAR images with deep learning
• Categorising the World into Local Climate Zones -- Towards Quantifying Labelling Uncertainty for Machine Learning Models
• SinkSAM-Net: Knowledge-Driven Self-Supervised Sinkhole Segmentation Using Topographic Priors and Segment Anything Model
• Zero-Shot Hyperspectral Pansharpening Using Hysteresis-Based Tuning for Spectral Quality Control
• A Multi-Task Deep Learning Framework for Building Footprint Segmentation
• CG-Net: Conditional GIS-aware Network for Individual Building Segmentation in VHR SAR Images
• On the Importance of 3D Surface Information for Remote Sensing Classification Tasks
• A High Resolution Urban and Rural Settlement Map of Africa Using Deep Learning and Satellite Imagery
• Uncertainties of Satellite-based Essential Climate Variables from Deep Learning
• A Mechanism-Learning Deeply Coupled Model for Remote Sensing Retrieval of Global Land Surface Temperature
• Integrating pre-processing pipelines in ODC based framework
• Learning and Interpreting Gravitational-Wave Features from CNNs with a Random Forest Approach
• Dual-Task Learning for Dead Tree Detection and Segmentation with Hybrid Self-Attention U-Nets in Aerial Imagery
• A deep multiple instance learning approach based on coarse labels for high-resolution land-cover mapping
• TimeSenCLIP: A Time Series Vision-Language Model for Remote Sensing
• ParkingTwin: Training-Free Streaming 3D Reconstruction for Parking-Lot Digital Twins
• LC-SLab -- An object-based deep learning framework for large-scale land cover classification from satellite imagery and sparse in-situ labels
• Adaptive Gradient Calibration for Single-Positive Multi-Label Learning in Remote Sensing Image Scene Classification