Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation

Abstract

In this work, we present a fully automatic method to segment cardiac structures from late-gadolinium enhanced (LGE) images without using labelled LGE data for training, but instead by transferring the anatomical knowledge and features learned on annotated balanced steady-state free precession (bSSFP) images, which are easier to acquire. Our framework mainly consists of two neural networks: a multi-modal image translation network for style transfer and a cascaded segmentation network for image segmentation. The multi-modal image translation network generates realistic and diverse synthetic LGE images conditioned on a single annotated bSSFP image, forming a synthetic LGE training set. This set is then utilized to fine-tune the segmentation network pre-trained on labelled bSSFP images, achieving the goal of unsupervised LGE image segmentation. In particular, the proposed cascaded segmentation network is able to produce accurate segmentation by taking both shape prior and image appearance into account, achieving an average Dice score of 0.92 for the left ventricle, 0.83 for the myocardium, and 0.88 for the right ventricle on the test set.

STACOM 2019 camera-ready. Winner of Multi-sequence Cardiac MR Segmentation Challenge (MS-CMRSeg 2019) https://zmiclab.github.io/mscmrseg19/

References in corpus (4)

• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Multivariate mixture model for myocardium segmentation combining multi-source images
• Multi-Task Learning for Left Atrial Segmentation on GE-MRI
• Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors

Cited by in corpus (14)

• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Learning Disentangled Representations in the Imaging Domain
• Disentangle, align and fuse for multimodal and semi-supervised image segmentation
• XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• Realistic Adversarial Data Augmentation for MR Image Segmentation
• 4D Semantic Cardiac Magnetic Resonance Image Synthesis on XCAT Anatomical Model
• ICMSC: Intra- and Cross-modality Semantic Consistency for Unsupervised Domain Adaptation on Hip Joint Bone Segmentation
• Style-invariant Cardiac Image Segmentation with Test-time Augmentation
• Semi-supervised Cardiac Image Segmentation via Label Propagation and Style Transfer
• Style Curriculum Learning for Robust Medical Image Segmentation
• Anatomically-Informed Deep Learning on Contrast-Enhanced Cardiac MRI for Scar Segmentation and Clinical Feature Extraction
• Adapt Everywhere: Unsupervised Adaptation of Point-Clouds and Entropy Minimisation for Multi-modal Cardiac Image Segmentation
• Machine Learning in Application to Automatic Noise Processing of Solar Spectrograms