Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery

Abstract

Change detection is one of the central problems in earth observation and was extensively investigated over recent decades. In this paper, we propose a novel recurrent convolutional neural network (ReCNN) architecture, which is trained to learn a joint spectral-spatial-temporal feature representation in a unified framework for change detection in multispectral images. To this end, we bring together a convolutional neural network (CNN) and a recurrent neural network (RNN) into one end-to-end network. The former is able to generate rich spectral-spatial feature representations, while the latter effectively analyzes temporal dependency in bi-temporal images. In comparison with previous approaches to change detection, the proposed network architecture possesses three distinctive properties: 1) It is end-to-end trainable, in contrast to most existing methods whose components are separately trained or computed; 2) it naturally harnesses spatial information that has been proven to be beneficial to change detection task; 3) it is capable of adaptively learning the temporal dependency between multitemporal images, unlike most of algorithms that use fairly simple operation like image differencing or stacking. As far as we know, this is the first time that a recurrent convolutional network architecture has been proposed for multitemporal remote sensing image analysis. The proposed network is validated on real multispectral data sets. Both visual and quantitative analysis of experimental results demonstrates competitive performance in the proposed mode.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep learning in remote sensing: a review
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Dense semantic labeling of sub-decimeter resolution images with convolutional neural networks
• Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders
• Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN
• Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification
• IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network

Cited by in corpus (35)

• Self-attention for raw optical Satellite Time Series Classification
• Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders
• Bi-Temporal Semantic Reasoning for the Semantic Change Detection in HR Remote Sensing Images
• Spatial-Spectral Feature Extraction via Deep ConvLSTM Neural Networks for Hyperspectral Image Classification
• Deep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection
• Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification
• Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images
• Change Guiding Network: Incorporating Change Prior to Guide Change Detection in Remote Sensing Imagery
• A3CLNN: Spatial, Spectral and Multiscale Attention ConvLSTM Neural Network for Multisource Remote Sensing Data Classification
• Joint Spatio-Temporal Modeling for the Semantic Change Detection in Remote Sensing Images
• Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification
• Mapping horizontal and vertical urban densification in Denmark with Landsat time-series from 1985 to 2018: a semantic segmentation solution
• Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning
• Hierarchical Paired Channel Fusion Network for Street Scene Change Detection
• From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data
• Building Footprint Generation by IntegratingConvolution Neural Network with Feature PairwiseConditional Random Field (FPCRF)
• A Survey of Sample-Efficient Deep Learning for Change Detection in Remote Sensing: Tasks, Strategies, and Challenges
• Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images
• Change Detection between Multimodal Remote Sensing Data Using Siamese CNN
• DSDANet: Deep Siamese Domain Adaptation Convolutional Neural Network for Cross-domain Change Detection
• Single-Temporal Supervised Learning for Universal Remote Sensing Change Detection
• Unsupervised Change Detection in Multi-temporal VHR Images Based on Deep Kernel PCA Convolutional Mapping Network
• Multitask Learning for Large-scale Semantic Change Detection
• An Attention-Based System for Damage Assessment Using Satellite Imagery
• Unsupervised Self-training Algorithm Based on Deep Learning for Optical Aerial Images Change Detection
• Enhanced SCanNet with CBAM and Dice Loss for Semantic Change Detection
• Towards Deep and Efficient: A Deep Siamese Self-Attention Fully Efficient Convolutional Network for Change Detection in VHR Images
• Multi-Modal Machine Learning for Flood Detection in News, Social Media and Satellite Sequences
• Semantic Change Detection with Asymmetric Siamese Networks
• XAI-Guided Enhancement of Vegetation Indices for Crop Mapping
• Space-Time-Aware Multi-Resolution Video Enhancement
• Synthetic Aperture Radar Image Change Detection via Siamese Adaptive Fusion Network
• Learning Woody Clearing With Loss Alignment for Zero-Shot Regrowth and Woody Segmentation
• VARENN: Graphical representation of spatiotemporal data and application to climate studies
• Task-Related Self-Supervised Learning for Remote Sensing Image Change Detection