A3CLNN: Spatial, Spectral and Multiscale Attention ConvLSTM Neural Network for Multisource Remote Sensing Data Classification

Abstract

The problem of effectively exploiting the information multiple data sources has become a relevant but challenging research topic in remote sensing. In this paper, we propose a new approach to exploit the complementarity of two data sources: hyperspectral images (HSIs) and light detection and ranging (LiDAR) data. Specifically, we develop a new dual-channel spatial, spectral and multiscale attention convolutional long short-term memory neural network (called dual-channel A3CLNN) for feature extraction and classification of multisource remote sensing data. Spatial, spectral and multiscale attention mechanisms are first designed for HSI and LiDAR data in order to learn spectral- and spatial-enhanced feature representations, and to represent multiscale information for different classes. In the designed fusion network, a novel composite attention learning mechanism (combined with a three-level fusion strategy) is used to fully integrate the features in these two data sources. Finally, inspired by the idea of transfer learning, a novel stepwise training strategy is designed to yield a final classification result. Our experimental results, conducted on several multisource remote sensing data sets, demonstrate that the newly proposed dual-channel A3CLNN exhibits better feature representation ability (leading to more competitive classification performance) than other state-of-the-art methods.

arXiv comment16 pages, 10 figures

References in corpus (8)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Squeeze-and-Excitation Networks
• Advances in Hyperspectral Image Classification: Earth monitoring with statistical learning methods
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Visual Saliency Based on Scale-Space Analysis in the Frequency Domain
• Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
• Spatial-Spectral Feature Extraction via Deep ConvLSTM Neural Networks for Hyperspectral Image Classification

Cited by in corpus (4)

• Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification
• Elevation Estimation-Driven Building 3D Reconstruction from Single-View Remote Sensing Imagery
• Hyperspectral and LiDAR data classification based on linear self-attention
• Auto Quantum Machine Learning for Multisource Classification