MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification

Abstract

Transformer has been extensively explored for hyperspectral image (HSI) classification. However, transformer poses challenges in terms of speed and memory usage because of its quadratic computational complexity. Recently, the Mamba model has emerged as a promising approach, which has strong long-distance modeling capabilities while maintaining a linear computational complexity. However, representing the HSI is challenging for the Mamba due to the requirement for an integrated spatial and spectral understanding. To remedy these drawbacks, we propose a novel HSI classification model based on a Mamba model, named MambaHSI, which can simultaneously model long-range interaction of the whole image and integrate spatial and spectral information in an adaptive manner. Specifically, we design a spatial Mamba block (SpaMB) to model the long-range interaction of the whole image at the pixel-level. Then, we propose a spectral Mamba block (SpeMB) to split the spectral vector into multiple groups, mine the relations across different spectral groups, and extract spectral features. Finally, we propose a spatial-spectral fusion module (SSFM) to adaptively integrate spatial and spectral features of a HSI. To our best knowledge, this is the first image-level HSI classification model based on the Mamba. We conduct extensive experiments on four diverse HSI datasets. The results demonstrate the effectiveness and superiority of the proposed model for HSI classification. This reveals the great potential of Mamba to be the next-generation backbone for HSI models. Codes are available at https://github.com/li-yapeng/MambaHSI .

accepted by IEEE TGRS

References in corpus (13)

• Graph Convolutional Networks for Hyperspectral Image Classification
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Hyperspectral Image Classification-Traditional to Deep Models: A Survey for Future Prospects
• Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
• A Fast 3D CNN for Hyperspectral Image Classification
• U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation
• VM-UNet: Vision Mamba UNet for Medical Image Segmentation
• A Spectral-Spatial-Dependent Global Learning Framework for Insufficient and Imbalanced Hyperspectral Image Classification
• Spectral-Spatial Mamba for Hyperspectral Image Classification
• Hungry Hungry Hippos: Towards Language Modeling with State Space Models
• Efficient Deep Learning of Non-local Features for Hyperspectral Image Classification
• SpectralMamba: Efficient Mamba for Hyperspectral Image Classification
• PointMamba: A Simple State Space Model for Point Cloud Analysis

Cited by in corpus (3)

• Multi-head Spatial-Spectral Mamba for Hyperspectral Image Classification
• Adaptive Multi-Order Graph Regularized NMF with Dual Sparsity for Hyperspectral Unmixing
• Hyperspectral Remote Sensing Images Salient Object Detection: The First Benchmark Dataset and Baseline