Classification of Hyperspectral and LiDAR Data Using Coupled CNNs

Abstract

In this paper, we propose an efficient and effective framework to fuse hyperspectral and Light Detection And Ranging (LiDAR) data using two coupled convolutional neural networks (CNNs). One CNN is designed to learn spectral-spatial features from hyperspectral data, and the other one is used to capture the elevation information from LiDAR data. Both of them consist of three convolutional layers, and the last two convolutional layers are coupled together via a parameter sharing strategy. In the fusion phase, feature-level and decision-level fusion methods are simultaneously used to integrate these heterogeneous features sufficiently. For the feature-level fusion, three different fusion strategies are evaluated, including the concatenation strategy, the maximization strategy, and the summation strategy. For the decision-level fusion, a weighted summation strategy is adopted, where the weights are determined by the classification accuracy of each output. The proposed model is evaluated on an urban data set acquired over Houston, USA, and a rural one captured over Trento, Italy. On the Houston data, our model can achieve a new record overall accuracy of 96.03%. On the Trento data, it achieves an overall accuracy of 99.12%. These results sufficiently certify the effectiveness of our proposed model.

References in corpus (4)

• Deep learning in remote sensing: a review
• An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing
• Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
• CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences

Cited by in corpus (15)

• Multimodal Fusion Transformer for Remote Sensing Image Classification
• Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution
• Spectral Superresolution of Multispectral Imagery with Joint Sparse and Low-Rank Learning
• Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification
• ASANet: Asymmetric Semantic Aligning Network for RGB and SAR image land cover classification
• Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications
• Single-stream CNN with Learnable Architecture for Multi-source Remote Sensing Data
• AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Framework
• Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion with Missing Data
• Hyperspectral Image Classification with Attention Aided CNNs
• HyperPointFormer: Multimodal Fusion in 3D Space with Dual-Branch Cross-Attention Transformers
• Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model
• FDCT: Frequency-Aware Decomposition and Cross-Modal Token-Alignment for Multi-Sensor Target Classification
• Hyperspectral and LiDAR data classification based on linear self-attention
• Spatial-Spectral Manifold Embedding of Hyperspectral Data