Deep Hyperspectral and Multispectral Image Fusion with Inter-image Variability
arXiv:2208.11376 · doi:10.1109/TGRS.2023.3273118
Abstract
Hyperspectral and multispectral image fusion allows us to overcome the hardware limitations of hyperspectral imaging systems inherent to their lower spatial resolution. Nevertheless, existing algorithms usually fail to consider realistic image acquisition conditions. This paper presents a general imaging model that considers inter-image variability of data from heterogeneous sources and flexible image priors. The fusion problem is stated as an optimization problem in the maximum a posteriori framework. We introduce an original image fusion method that, on the one hand, solves the optimization problem accounting for inter-image variability with an iteratively reweighted scheme and, on the other hand, that leverages light-weight CNN-based networks to learn realistic image priors from data. In addition, we propose a zero-shot strategy to directly learn the image-specific prior of the latent images in an unsupervised manner. The performance of the algorithm is illustrated with real data subject to inter-image variability.
arXiv commentIEEE Trans. Geosci. Remote sens., to be published. Manuscript submitted August 23, 2022; revised Dec. 15, 2022, and Mar. 13, 2023; and accepted Apr. 07, 2023
References in corpus (18)
- Adam: A Method for Stochastic Optimization
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Tensor Decomposition for Signal Processing and Machine Learning
- A convex formulation for hyperspectral image superresolution via subspace-based regularization
- Hyperspectral and Multispectral Image Fusion based on a Sparse Representation
- Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation
- Nonlinear unmixing of hyperspectral images: models and algorithms
- Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network
- Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review
- Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach
- Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution
- Hyperspectral Super-Resolution with Coupled Tucker Approximation: Recoverability and SVD-based algorithms
- Hyperspectral Super-Resolution via Interpretable Block-Term Tensor Modeling
- Integration of Physics-Based and Data-Driven Models for Hyperspectral Image Unmixing
- Coupled Tensor Decomposition for Hyperspectral and Multispectral Image Fusion with Inter-Image Variability
- Blind and fully constrained unmixing of hyperspectral images
- A New Adaptive Video Super-Resolution Algorithm With Improved Robustness to Innovations
- Tuning-free Plug-and-Play Hyperspectral Image Deconvolution with Deep Priors