DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions. Prior methods suffer from limited representation ability, as they train specially designed networks from scratch on limited annotated data. We propose a tri-spectral image generation pipeline that transforms HSI into high-quality tri-spectral images, enabling the use of off-the-shelf ImageNet pretrained backbone networks for feature extraction. Motivated by the observation that there are many homogeneous areas with distinguished semantic and geometric properties in HSIs, which can be used to extract useful contexts, we propose an end-to-end segmentation network named DCN-T. It adopts transformers to effectively encode regional adaptation and global aggregation spatial contexts within and between the homogeneous areas discovered by similarity-based clustering. To fully exploit the rich spectrums of the HSI, we adopt an ensemble approach where all segmentation results of the tri-spectral images are integrated into the final prediction through a voting scheme. Extensive experiments on three public benchmarks show that our proposed method outperforms state-of-the-art methods for HSI classification.

arXiv commentAccepted by IEEE TIP. The code will be released at https://github.com/DotWang/DCN-T

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Remote Sensing Image Scene Classification: Benchmark and State of the Art
• PVT v2: Improved Baselines with Pyramid Vision Transformer
• Graph Convolutional Networks for Hyperspectral Image Classification
• Going Deeper with Contextual CNN for Hyperspectral Image Classification
• Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
• Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review
• An Empirical Study of Remote Sensing Pretraining
• Spatial-Spectral Feature Extraction via Deep ConvLSTM Neural Networks for Hyperspectral Image Classification
• FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification
• ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias
• BASS Net: Band-Adaptive Spectral-Spatial Feature Learning Neural Network for Hyperspectral Image Classification
• Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers
• Efficient Deep Learning of Non-local Features for Hyperspectral Image Classification

Cited by in corpus (1)

• Dual-stage Hyperspectral Image Classification Model with Spectral Supertoken