Semisupervised Manifold Alignment of Multimodal Remote Sensing Images

Abstract

We introduce a method for manifold alignment of different modalities (or domains) of remote sensing images. The problem is recurrent when a set of multitemporal, multisource, multisensor and multiangular images is available. In these situations, images should ideally be spatially coregistred, corrected and compensated for differences in the image domains. Such procedures require the interaction of the user, involve tuning of many parameters and heuristics, and are usually applied separately. Changes of sensors and acquisition conditions translate into shifts, twists, warps and foldings of the image distributions (or manifolds). The proposed semisupervised manifold alignment (SS-MA) method aligns the images working directly on their manifolds, and is thus not restricted to images of the same resolutions, either spectral or spatial. SS-MA pulls close together samples of the same class while pushing those of different classes apart. At the same time, it preserves the geometry of each manifold along the transformation. The method builds a linear invertible transformation to a latent space where all images are alike, and reduces to solving a generalized eigenproblem of moderate size. We study the performance of SS-MA in toy examples and in real multiangular, multitemporal, and multisource image classification problems. The method performs well for strong deformations and leads to accurate classification for all domains.

References in corpus (2)

• A survey of active learning algorithms for supervised remote sensing image classification
• Kernel Multivariate Analysis Framework for Supervised Subspace Learning: A Tutorial on Linear and Kernel Multivariate Methods

Cited by in corpus (18)

• More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• Recent Advances in Domain Adaptation for the Classification of Remote Sensing Data
• Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data
• CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences
• Understanding urban landuse from the above and ground perspectives: a deep learning, multimodal solution
• Kernel Manifold Alignment
• MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data
• Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study
• Multi-temporal and multi-source remote sensing image classification by nonlinear relative normalization
• Learning Shared Cross-modality Representation Using Multispectral-LiDAR and Hyperspectral Data
• Multisource Region Attention Network for Fine-Grained Object Recognition in Remote Sensing Imagery
• Cross-Domain Collaborative Learning via Cluster Canonical Correlation Analysis and Random Walker for Hyperspectral Image Classification
• Tensor Alignment Based Domain Adaptation for Hyperspectral Image Classification
• Non-convex regularization in remote sensing
• Weakly Supervised Instance Attention for Multisource Fine-Grained Object Recognition with an Application to Tree Species Classification
• Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V images for Cloud Detection
• Domain-Adversarial Training of Self-Attention Based Networks for Land Cover Classification using Multi-temporal Sentinel-2 Satellite Imagery