Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification

Abstract

In this paper, we aim at tackling a general but interesting cross-modality feature learning question in remote sensing community --- can a limited amount of highly-discrimin-ative (e.g., hyperspectral) training data improve the performance of a classification task using a large amount of poorly-discriminative (e.g., multispectral) data? Traditional semi-supervised manifold alignment methods do not perform sufficiently well for such problems, since the hyperspectral data is very expensive to be largely collected in a trade-off between time and efficiency, compared to the multispectral data. To this end, we propose a novel semi-supervised cross-modality learning framework, called learnable manifold alignment (LeMA). LeMA learns a joint graph structure directly from the data instead of using a given fixed graph defined by a Gaussian kernel function. With the learned graph, we can further capture the data distribution by graph-based label propagation, which enables finding a more accurate decision boundary. Additionally, an optimization strategy based on the alternating direction method of multipliers (ADMM) is designed to solve the proposed model. Extensive experiments on two hyperspectral-multispectral datasets demonstrate the superiority and effectiveness of the proposed method in comparison with several state-of-the-art methods.

References in corpus (8)

• Robust Recovery of Subspace Structures by Low-Rank Representation
• The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices
• Recent Advances in Domain Adaptation for the Classification of Remote Sensing Data
• Building Instance Classification Using Street View Images
• Semisupervised Manifold Alignment of Multimodal Remote Sensing Images
• Multi-temporal and multi-source remote sensing image classification by nonlinear relative normalization
• Canonical Correlation Forests
• On the Equivalence of Inexact Proximal ALM and ADMM for a Class of Convex Composite Programming

Cited by in corpus (32)

• Graph Convolutional Networks for Hyperspectral Image Classification
• More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)
• Hyperspectral Image Classification-Traditional to Deep Models: A Survey for Future Prospects
• Deep Learning for UAV-based Object Detection and Tracking: A Survey
• Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification
• X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for Classification of Remote Sensing Data
• ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features
• Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution
• Spectral Superresolution of Multispectral Imagery with Joint Sparse and Low-Rank Learning
• A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models
• A framework for large-scale mapping of human settlement extent from Sentinel-2 images via fully convolutional neural networks
• MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data
• Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study
• Learning Convolutional Sparse Coding on Complex Domain for Interferometric Phase Restoration
• Learning Shared Cross-modality Representation Using Multispectral-LiDAR and Hyperspectral Data
• Vehicle Detection of Multi-source Remote Sensing Data Using Active Fine-tuning Network
• Weakly Supervised Instance Attention for Multisource Fine-Grained Object Recognition with an Application to Tree Species Classification
• Multipass SAR Interferometry Based on Total Variation Regularized Robust Low Rank Tensor Decomposition
• Rotation Equivariant Feature Image Pyramid Network for Object Detection in Optical Remote Sensing Imagery
• Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion with Missing Data
• Distributed Learning over Networks with Graph-Attention-Based Personalization
• Triplet-Watershed for Hyperspectral Image Classification
• Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution
• Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model
• Divergence Regulated Encoder Network for Joint Dimensionality Reduction and Classification
• Joint and Progressive Subspace Analysis (JPSA) with Spatial-Spectral Manifold Alignment for Semi-Supervised Hyperspectral Dimensionality Reduction
• Cross-Modal and Multimodal Data Analysis Based on Functional Mapping of Spectral Descriptors and Manifold Regularization
• Multi-level Feature Fusion-based CNN for Local Climate Zone Classification from Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset
• Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images
• Single-Look Multi-Master SAR Tomography: An Introduction
• Spatial-Spectral Manifold Embedding of Hyperspectral Data