Missing Data Reconstruction in Remote Sensing image with a Unified Spatial-Temporal-Spectral Deep Convolutional Neural Network

Abstract

Because of the internal malfunction of satellite sensors and poor atmospheric conditions such as thick cloud, the acquired remote sensing data often suffer from missing information, i.e., the data usability is greatly reduced. In this paper, a novel method of missing information reconstruction in remote sensing images is proposed. The unified spatial-temporal-spectral framework based on a deep convolutional neural network (STS-CNN) employs a unified deep convolutional neural network combined with spatial-temporal-spectral supplementary information. In addition, to address the fact that most methods can only deal with a single missing information reconstruction task, the proposed approach can solve three typical missing information reconstruction tasks: 1) dead lines in Aqua MODIS band 6; 2) the Landsat ETM+ Scan Line Corrector (SLC)-off problem; and 3) thick cloud removal. It should be noted that the proposed model can use multi-source data (spatial, spectral, and temporal) as the input of the unified framework. The results of both simulated and real-data experiments demonstrate that the proposed model exhibits high effectiveness in the three missing information reconstruction tasks listed above.

To be published in IEEE Transactions on Geoscience and Remote Sensing

References in corpus (4)

• AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification
• Boosting the accuracy of multi-spectral image pan-sharpening by learning a deep residual network
• Embedding Structured Contour and Location Prior in Siamesed Fully Convolutional Networks for Road Detection
• A Joint Convolutional Neural Networks and Context Transfer for Street Scenes Labeling

Cited by in corpus (14)

• Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network
• Deep Learning Inversion of Electrical Resistivity Data
• Hybrid Noise Removal in Hyperspectral Imagery With a Spatial-Spectral Gradient Network
• Spatially Continuous and High-resolution Land Surface Temperature: A Review of Reconstruction and Spatiotemporal Fusion Techniques
• Spatial-Spectral Fusion by Combining Deep Learning and Variation Model
• A lightweight deep learning based cloud detection method for Sentinel-2A imagery fusing multi-scale spectral and spatial features
• Bilevel Optimization, Deep Learning and Fractional Laplacian Regularization with Applications in Tomography
• IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images
• Multi-scale data reconstruction of turbulent rotating flows with Gappy POD, Extended POD and Generative Adversarial Networks
• Measuring Human and Economic Activity from Satellite Imagery to Support City-Scale Decision-Making during COVID-19 Pandemic
• An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images
• Generative Adversarial Networks to infer velocity components in rotating turbulent flows
• Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network
• Thick Cloud Removal of Remote Sensing Images Using Temporal Smoothness and Sparsity-Regularized Tensor Optimization