SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Abstract

Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the contrast and resolution of the synthetic training data. Consequently, SynthSeg can segment real scans from a wide range of target domains without retraining or fine-tuning, which enables straightforward analysis of huge amounts of heterogeneous clinical data. Because SynthSeg only requires segmentations to be trained (no images), it can learn from labels obtained by automated methods on diverse populations (e.g., ageing and diseased), thus achieving robustness to a wide range of morphological variability. We demonstrate SynthSeg on 5,000 scans of six modalities (including CT) and ten resolutions, where it exhibits unparalleled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation. Finally, we demonstrate the generalisability of SynthSeg by applying it to cardiac MRI and CT scans.

References in corpus (3)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Direct Estimation of Spinal Cobb Angles by Structured Multi-Output Regression

Cited by in corpus (36)

• Domain Generalization for Medical Image Analysis: A Review
• Anatomy-aware and acquisition-agnostic joint registration with SynthMorph
• Towards contrast-agnostic soft segmentation of the spinal cord
• Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge
• FastSurfer-HypVINN: Automated sub-segmentation of the hypothalamus and adjacent structures on high-resolutional brain MRI
• Boosting Skull-Stripping Performance for Pediatric Brain Images
• CHeart: A Conditional Spatio-Temporal Generative Model for Cardiac Anatomy
• IGUANe: a 3D generalizable CycleGAN for multicenter harmonization of brain MR images
• The TopCoW Challenge -- Topology-Aware Circle of Willis Segmentation for CT and MR Angiography
• Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis
• Weakly Supervised Intracranial Hemorrhage Segmentation using Head-Wise Gradient-Infused Self-Attention Maps from a Swin Transformer in Categorical Learning
• SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI
• Supervision by Denoising for Medical Image Segmentation
• A voxel-level approach to brain age prediction: A method to assess regional brain aging
• Domain-agnostic segmentation of thalamic nuclei from joint structural and diffusion MRI
• A deformation-based morphometry framework for disentangling Alzheimer's disease from normal aging using learned normal aging templates
• Advances in Automated Fetal Brain MRI Segmentation and Biometry: Insights from the FeTA 2024 Challenge
• Synthesizing Individualized Aging Brains in Health and Disease with Generative Models and Parallel Transport
• Learning accurate rigid registration for longitudinal brain MRI from synthetic data
• SCIsegV2: A Universal Tool for Segmentation of Intramedullary Lesions in Spinal Cord Injury
• UNISELF: A Unified Network with Instance Normalization and Self-Ensembled Lesion Fusion for Multiple Sclerosis Lesion Segmentation
• Towards contrast- and pathology-agnostic clinical fetal brain MRI segmentation using SynthSeg
• Domain-randomized deep learning for neuroimage analysis
• SynthBA: Reliable Brain Age Estimation Across Multiple MRI Sequences and Resolutions
• Automated neuroradiological support systems for multiple cerebrovascular disease markers -- A systematic review and meta-analysis
• Impact of Initialization on Intra-subject Pediatric Brain MR Image Registration: A Comparative Analysis between SyN ANTs and Deep Learning-Based Approaches
• Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantification
• Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts
• SingleStrip: learning skull-stripping from a single labeled example
• Deep infant brain segmentation from multi-contrast MRI
• Segmenting infant brains across magnetic fields: Domain randomization and annotation curation in ultra-low field MRI
• Impact of Synthetic Lesional MR Images in Automated Focal Cortical Dysplasia Detection in Low-Data Scenarios
• Structural MRI Synthesis for Alzheimer's Disease via Conditional Diffusion on Anatomical Masks
• Subcortical Masks Generation in CT Images via Ensemble-Based Cross-Domain Label Transfer
• Enhancing Corpus Callosum Segmentation in Fetal MRI via Pathology-Informed Domain Randomization
• Deep Biomechanically-Guided Interpolation for Keypoint-Based Brain Shift Registration