SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models

Abstract

Hyperspectral Image (HSI) classification is an important issue in remote sensing field with extensive applications in earth science. In recent years, a large number of deep learning-based HSI classification methods have been proposed. However, existing methods have limited ability to handle high-dimensional, highly redundant, and complex data, making it challenging to capture the spectral-spatial distributions of data and relationships between samples. To address this issue, we propose a generative framework for HSI classification with diffusion models (SpectralDiff) that effectively mines the distribution information of high-dimensional and highly redundant data by iteratively denoising and explicitly constructing the data generation process, thus better reflecting the relationships between samples. The framework consists of a spectral-spatial diffusion module, and an attention-based classification module. The spectral-spatial diffusion module adopts forward and reverse spectral-spatial diffusion processes to achieve adaptive construction of sample relationships without requiring prior knowledge of graphical structure or neighborhood information. It captures spectral-spatial distribution and contextual information of objects in HSI and mines unsupervised spectral-spatial diffusion features within the reverse diffusion process. Finally, these features are fed into the attention-based classification module for per-pixel classification. The diffusion features can facilitate cross-sample perception via reconstruction distribution, leading to improved classification performance. Experiments on three public HSI datasets demonstrate that the proposed method can achieve better performance than state-of-the-art methods. For the sake of reproducibility, the source code of SpectralDiff will be publicly available at https://github.com/chenning0115/SpectralDiff.

References in corpus (20)

• Denoising Diffusion Probabilistic Models
• Deep learning in remote sensing: a review
• Diffusion Models Beat GANs on Image Synthesis
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Deep Learning for Hyperspectral Image Classification: An Overview
• Diffusion Models in Vision: A Survey
• Graph Convolutional Networks for Hyperspectral Image Classification
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Score-Based Generative Modeling through Stochastic Differential Equations
• Generative Modeling by Estimating Gradients of the Data Distribution
• Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)
• Variational Diffusion Models
• Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning
• Speech Enhancement and Dereverberation with Diffusion-based Generative Models
• Torsional Diffusion for Molecular Conformer Generation
• Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives
• EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations
• Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models
• A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Material
• Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models

Cited by in corpus (5)

• A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models
• Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives
• Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives
• BSDM: Background Suppression Diffusion Model for Hyperspectral Anomaly Detection
• Stereo Event-based, 6-DOF Pose Tracking for Uncooperative Spacecraft