DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation

Abstract

Deep convolutional neural networks (CNNs) are state-of-the-art for semantic image segmentation, but typically require many labeled training samples. Obtaining 3D segmentations of medical images for supervised training is difficult and labor intensive. Motivated by classical approaches for joint segmentation and registration we therefore propose a deep learning framework that jointly learns networks for image registration and image segmentation. In contrast to previous work on deep unsupervised image registration, which showed the benefit of weak supervision via image segmentations, our approach can use existing segmentations when available and computes them via the segmentation network otherwise, thereby providing the same registration benefit. Conversely, segmentation network training benefits from the registration, which essentially provides a realistic form of data augmentation. Experiments on knee and brain 3D magnetic resonance (MR) images show that our approach achieves large simultaneous improvements of segmentation and registration accuracy (over independently trained networks) and allows training high-quality models with very limited training data. Specifically, in a one-shot-scenario (with only one manually labeled image) our approach increases Dice scores (%) over an unsupervised registration network by 2.7 and 1.8 on the knee and brain images respectively.

arXiv commentTo appear in MICCAI 2019

References in corpus (1)

• A Survey on Deep Learning in Medical Image Analysis

Cited by in corpus (7)

• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Longitudinal diffusion MRI analysis using Segis-Net: a single-step deep-learning framework for simultaneous segmentation and registration
• A Cross-Stitch Architecture for Joint Registration and Segmentation in Adaptive Radiotherapy
• Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation
• Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Images
• Multi-task learning for joint weakly-supervised segmentation and aortic arch anomaly classification in fetal cardiac MRI
• Anatomical Data Augmentation via Fluid-based Image Registration