Label-driven weakly-supervised learning for multimodal deformable image registration

Abstract

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a cross-entropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fully-automatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global- and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.

Accepted to ISBI 2018

References in corpus (2)

• End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
• Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks

Cited by in corpus (12)

• VoxelMorph: A Learning Framework for Deformable Medical Image Registration
• A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration
• Deep Learning in Medical Image Registration: A Survey
• Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration
• Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces
• SynthMorph: learning contrast-invariant registration without acquired images
• Medical image registration using unsupervised deep neural network: A scoping literature review
• Adversarial Deformation Regularization for Training Image Registration Neural Networks
• mlVIRNET: Multilevel Variational Image Registration Network
• DeepReg: a deep learning toolkit for medical image registration
• Longitudinal diffusion MRI analysis using Segis-Net: a single-step deep-learning framework for simultaneous segmentation and registration
• The use of deep learning in interventional radiotherapy (brachytherapy): a review with a focus on open source and open data