Deep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection

Abstract

Image translation with convolutional neural networks has recently been used as an approach to multimodal change detection. Existing approaches train the networks by exploiting supervised information of the change areas, which, however, is not always available. A main challenge in the unsupervised problem setting is to avoid that change pixels affect the learning of the translation function. We propose two new network architectures trained with loss functions weighted by priors that reduce the impact of change pixels on the learning objective. The change prior is derived in an unsupervised fashion from relational pixel information captured by domain-specific affinity matrices. Specifically, we use the vertex degrees associated with an absolute affinity difference matrix and demonstrate their utility in combination with cycle consistency and adversarial training. The proposed neural networks are compared with state-of-the-art algorithms. Experiments conducted on three real datasets show the effectiveness of our methodology.

References in corpus (8)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep learning in remote sensing: a review
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• On the Convergence of Adam and Beyond
• More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery
• Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)
• Unsupervised Image Regression for Heterogeneous Change Detection

Cited by in corpus (8)

• Bi-Temporal Semantic Reasoning for the Semantic Change Detection in HR Remote Sensing Images
• Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning
• A Survey of Sample-Efficient Deep Learning for Change Detection in Remote Sensing: Tasks, Strategies, and Challenges
• Transformer-based Multimodal Change Detection with Multitask Consistency Constraints
• NN-Copula-CD: A Copula-Guided Interpretable Neural Network for Change Detection in Heterogeneous Remote Sensing Images
• Recognition of polar lows in Sentinel-1 SAR images with deep learning
• Federated CycleGAN for Privacy-Preserving Image-to-Image Translation
• Self-supervised Change Detection in Multi-view Remote Sensing Images