Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

Abstract

Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solution is based on the cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists and bridge the domain gap in radiological images. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups. Built upon adversarial training, we propose a learnable self-attentive spatial normalization of the deep convolutional generator network's intermediate activations. Unlike previous attention-based image-to-image translation approaches, which are either domain-specific or require distortion of the source domain's structures, we unearth the importance of the auxiliary semantic information to handle the geometric changes and preserve anatomical structures during image translation. We achieve superior results for cross-modality segmentation between unpaired MRI and CT data for multi-modality whole heart and multi-modal brain tumor MRI (T1/T2) datasets compared to the state-of-the-art methods. We also observe encouraging results in cross-modality conversion for paired MRI and CT images on a brain dataset. Furthermore, a detailed analysis of the cross-modality image translation, thorough ablation studies confirm our proposed method's efficacy.

Accepted for publication in IEEE Transactions on Medical Imaging (IEEE TMI)

References in corpus (4)

• GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification
• Automatic Liver Lesion Segmentation Using A Deep Convolutional Neural Network Method
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis

Cited by in corpus (4)

• Is attention all you need in medical image analysis? A review
• FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation
• An Attentive-based Generative Model for Medical Image Synthesis
• Fully Differentiable Bidirectional Dual-Task Synergistic Learning for Semi-Supervised 3D Medical Image Segmentation