MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data

Abstract

Multi-modal data fusion has recently been shown promise in classification tasks in remote sensing. Optical data and radar data, two important yet intrinsically different data sources, are attracting more and more attention for potential data fusion. It is already widely known that, a machine learning based methodology often yields excellent performance. However, the methodology relies on a large training set, which is very expensive to achieve in remote sensing. The semi-supervised manifold alignment (SSMA), a multi-modal data fusion algorithm, has been designed to amplify the impact of an existing training set by linking labeled data to unlabeled data via unsupervised techniques. In this paper, we explore the potential of SSMA in fusing optical data and polarimetric SAR data, which are multi-sensory data sources. Furthermore, we propose a MAPPER-induced manifold alignment (MIMA) for semi-supervised fusion of multi-sensory data sources. Our proposed method unites SSMA with MAPPER, which is developed from the emerging topological data analysis (TDA) field. To our best knowledge, this is the first time that SSMA has been applied on fusing optical data and SAR data, and also the first time that TDA has been applied in remote sensing. The conventional SSMA derives a topological structure using k-nearest-neighbor (kNN), while MIMA employs MAPPER, which considers the field knowledge and derives a novel topological structure through the spectral clustering in a data-driven fashion. Experiment results on data fusion with respect to land cover land use classification and local climate zone classification suggest superior performance of MIMA.

References in corpus (9)

• An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing
• Cascaded Recurrent Neural Networks for Hyperspectral Image Classification
• Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification
• Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN
• CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences
• ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features
• Semisupervised Manifold Alignment of Multimodal Remote Sensing Images
• Multi-temporal and multi-source remote sensing image classification by nonlinear relative normalization
• Statistical Analysis and Parameter Selection for Mapper

Cited by in corpus (9)

• More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data
• Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution
• Spectral Superresolution of Multispectral Imagery with Joint Sparse and Low-Rank Learning
• SAR Image Change Detection Based on Multiscale Capsule Network
• SAR Image Change Detection Based on Multiscale Capsule Network
• Single-stream CNN with Learnable Architecture for Multi-source Remote Sensing Data
• Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model
• Evaluating Self and Semi-Supervised Methods for Remote Sensing Segmentation Tasks