Few-Shot Hyperspectral Image Classification With Unknown Classes Using Multitask Deep Learning

Abstract

Current hyperspectral image classification assumes that a predefined classification system is closed and complete, and there are no unknown or novel classes in the unseen data. However, this assumption may be too strict for the real world. Often, novel classes are overlooked when the classification system is constructed. The closed nature forces a model to assign a label given a new sample and may lead to overestimation of known land covers (e.g., crop area). To tackle this issue, we propose a multitask deep learning method that simultaneously conducts classification and reconstruction in the open world (named MDL4OW) where unknown classes may exist. The reconstructed data are compared with the original data; those failing to be reconstructed are considered unknown, based on the assumption that they are not well represented in the latent features due to the lack of labels. A threshold needs to be defined to separate the unknown and known classes; we propose two strategies based on the extreme value theory for few-shot and many-shot scenarios. The proposed method was tested on real-world hyperspectral images; state-of-the-art results were achieved, e.g., improving the overall accuracy by 4.94% for the Salinas data. By considering the existence of unknown classes in the open world, our method achieved more accurate hyperspectral image classification, especially under the few-shot context.

Accepted by IEEE TGRS

References in corpus (2)

• Deep Learning for Classification of Hyperspectral Data: A Comparative Review
• Locality and Structure Regularized Low Rank Representation for Hyperspectral Image Classification

Cited by in corpus (7)

• SR2CNN: Zero-Shot Learning for Signal Recognition
• SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models
• Robust Self-Ensembling Network for Hyperspectral Image Classification
• Nonnegative-Constrained Joint Collaborative Representation with Union Dictionary for Hyperspectral Anomaly Detection
• A fast and Accurate Similarity-constrained Subspace Clustering Framework for Unsupervised Hyperspectral Image Classification
• Transformer-Driven Active Transfer Learning for Cross-Hyperspectral Image Classification
• Deep Feature Gaussian Processes for Single-Scene Aerosol Optical Depth Reconstruction