Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution

Abstract

Due to the limitations of hyperspectral imaging systems, hyperspectral imagery (HSI) often suffers from poor spatial resolution, thus hampering many applications of the imagery. Hyperspectral super-resolution refers to fusing HSI and MSI to generate an image with both high spatial and high spectral resolutions. Recently, several new methods have been proposed to solve this fusion problem, and most of these methods assume that the prior information of the Point Spread Function (PSF) and Spectral Response Function (SRF) are known. However, in practice, this information is often limited or unavailable. In this work, an unsupervised deep learning-based fusion method - HyCoNet - that can solve the problems in HSI-MSI fusion without the prior PSF and SRF information is proposed. HyCoNet consists of three coupled autoencoder nets in which the HSI and MSI are unmixed into endmembers and abundances based on the linear unmixing model. Two special convolutional layers are designed to act as a bridge that coordinates with the three autoencoder nets, and the PSF and SRF parameters are learned adaptively in the two convolution layers during the training process. Furthermore, driven by the joint loss function, the proposed method is straightforward and easily implemented in an end-to-end training manner. The experiments performed in the study demonstrate that the proposed method performs well and produces robust results for different datasets and arbitrary PSFs and SRFs.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing
• A convex formulation for hyperspectral image superresolution via subspace-based regularization
• Hyperspectral and Multispectral Image Fusion based on a Sparse Representation
• Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)
• Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation
• Classification of Hyperspectral and LiDAR Data Using Coupled CNNs
• Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network
• Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification
• Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification
• ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features
• Fourier-based Rotation-invariant Feature Boosting: An Efficient Framework for Geospatial Object Detection
• MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data
• Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration
• Learning Shared Cross-modality Representation Using Multispectral-LiDAR and Hyperspectral Data

Cited by in corpus (7)

• Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution
• Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Net
• A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution
• Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy
• Unmixing based PAN guided fusion network for hyperspectral imagery
• Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution
• Spectral Image Data Fusion for Multisource Data Augmentation