Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

Abstract

Earth observation (EO) sensors deliver data with daily or weekly temporal resolution. Most land use and land cover (LULC) approaches, however, expect cloud-free and mono-temporal observations. The increasing temporal capabilities of today's sensors enables the use of temporal, along with spectral and spatial features. Domains, such as speech recognition or neural machine translation, work with inherently temporal data and, today, achieve impressive results using sequential encoder-decoder structures. Inspired by these sequence-to-sequence models, we adapt an encoder structure with convolutional recurrent layers in order to approximate a phenological model for vegetation classes based on a temporal sequence of Sentinel 2 (S2) images. In our experiments, we visualize internal activations over a sequence of cloudy and non-cloudy images and find several recurrent cells, which reduce the input activity for cloudy observations. Hence, we assume that our network has learned cloud-filtering schemes solely from input data, which could alleviate the need for tedious cloud-filtering as a preprocessing step for many EO approaches. Moreover, using unfiltered temporal series of top-of-atmosphere (TOA) reflectance data, we achieved in our experiments state-of-the-art classification accuracies on a large number of crop classes with minimal preprocessing compared to other classification approaches.

References in corpus (3)

• Deep learning in remote sensing: a review
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Land Cover Classification from Multi-temporal, Multi-spectral Remotely Sensed Imagery using Patch-Based Recurrent Neural Networks

Cited by in corpus (16)

• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Self-attention for raw optical Satellite Time Series Classification
• Spatial-Spectral Feature Extraction via Deep ConvLSTM Neural Networks for Hyperspectral Image Classification
• Land Cover Classification from Remote Sensing Images Based on Multi-Scale Fully Convolutional Network
• Improvement in Land Cover and Crop Classification based on Temporal Features Learning from Sentinel-2 Data Using Recurrent-Convolutional Neural Network (R-CNN)
• Satellite Image Time Series Analysis for Big Earth Observation Data
• A framework for large-scale mapping of human settlement extent from Sentinel-2 images via fully convolutional neural networks
• TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation
• Context-self contrastive pretraining for crop type semantic segmentation
• A systematic review of the use of Deep Learning in Satellite Imagery for Agriculture
• Location-aware Adaptive Normalization: A Deep Learning Approach For Wildfire Danger Forecasting
• Cloud gap-filling with deep learning for improved grassland monitoring
• 3D Fully Convolutional Neural Networks with Intersection Over Union Loss for Crop Mapping from Multi-Temporal Satellite Images
• Tree semantic segmentation from aerial image time series
• Harnessing Administrative Data Inventories to Create a Reliable Transnational Reference Database for Crop Type Monitoring
• TimeSenCLIP: A Time Series Vision-Language Model for Remote Sensing