Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN

Abstract

In this letter, we propose a pseudo-siamese convolutional neural network (CNN) architecture that enables to solve the task of identifying corresponding patches in very-high-resolution (VHR) optical and synthetic aperture radar (SAR) remote sensing imagery. Using eight convolutional layers each in two parallel network streams, a fully connected layer for the fusion of the features learned in each stream, and a loss function based on binary cross-entropy, we achieve a one-hot indication if two patches correspond or not. The network is trained and tested on an automatically generated dataset that is based on a deterministic alignment of SAR and optical imagery via previously reconstructed and subsequently co-registered 3D point clouds. The satellite images, from which the patches comprising our dataset are extracted, show a complex urban scene containing many elevated objects (i.e. buildings), thus providing one of the most difficult experimental environments. The achieved results show that the network is able to predict corresponding patches with high accuracy, thus indicating great potential for further development towards a generalized multi-sensor key-point matching procedure. Index Terms-synthetic aperture radar (SAR), optical imagery, data fusion, deep learning, convolutional neural networks (CNN), image matching, deep matching

References in corpus (3)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity

Cited by in corpus (15)

• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Building Instance Classification Using Street View Images
• Buildings Detection in VHR SAR Images Using Fully Convolution Neural Networks
• Local Feature Matching Using Deep Learning: A Survey
• MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data
• Backdoor Pre-trained Models Can Transfer to All
• Detecting quantum entanglement with unsupervised learning
• AeroRIT: A New Scene for Hyperspectral Image Analysis
• Multimodal Representation for Neural Code Search
• The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion
• MPSN: Motion-aware Pseudo Siamese Network for Indoor Video Head Detection in Buildings
• The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion
• Improving Deep Learning Performance for Predicting Large-Scale Porous-Media Flow through Feature Coarsening
• COMPOSE: Cross-Modal Pseudo-Siamese Network for Patient Trial Matching
• Boosting ship detection in SAR images with complementary pretraining techniques