Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation

Abstract

This paper presents a novel unsupervised domain adaptation framework, called Synergistic Image and Feature Adaptation (SIFA), to effectively tackle the problem of domain shift. Domain adaptation has become an important and hot topic in recent studies on deep learning, aiming to recover performance degradation when applying the neural networks to new testing domains. Our proposed SIFA is an elegant learning diagram which presents synergistic fusion of adaptations from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features towards the segmentation task. The feature encoder layers are shared by both perspectives to grasp their mutual benefits during the end-to-end learning procedure. Without using any annotation from the target domain, the learning of our unified model is guided by adversarial losses, with multiple discriminators employed from various aspects. We have extensively validated our method with a challenging application of cross-modality medical image segmentation of cardiac structures. Experimental results demonstrate that our SIFA model recovers the degraded performance from 17.2% to 73.0%, and outperforms the state-of-the-art methods by a significant margin.

arXiv commentAAAI 2019 (oral)

References in corpus (4)

• Learning Transferable Features with Deep Adaptation Networks
• Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation
• From source to target and back: symmetric bi-directional adaptive GAN
• Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation

Cited by in corpus (9)

• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets
• Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation
• PSIGAN: Joint probabilistic segmentation and image distribution matching for unpaired cross-modality adaptation based MRI segmentation
• A Survey on Deep Learning of Small Sample in Biomedical Image Analysis
• Domain Adaptive Medical Image Segmentation via Adversarial Learning of Disease-Specific Spatial Patterns
• Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation
• Data Efficient Unsupervised Domain Adaptation for Cross-Modality Image Segmentation
• Multi-sequence Cardiac MR Segmentation with Adversarial Domain Adaptation Network