A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration

Abstract

Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for \textit{unsupervised} affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.

Accepted: Medical Image Analysis - Elsevier

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Fully Convolutional Networks for Semantic Segmentation
• FlowNet: Learning Optical Flow with Convolutional Networks
• Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration
• End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
• Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Label-driven weakly-supervised learning for multimodal deformable image registration
• Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness

Cited by in corpus (92)

• An overview of deep learning in medical imaging focusing on MRI
• Recent advances and clinical applications of deep learning in medical image analysis
• TransMorph: Transformer for unsupervised medical image registration
• Deep Learning in Medical Image Registration: A Review
• Deep Learning in Medical Image Registration: A Survey
• Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces
• Recursive Cascaded Networks for Unsupervised Medical Image Registration
• Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network
• SynthMorph: learning contrast-invariant registration without acquired images
• Medical Image Registration Using Deep Neural Networks: A Comprehensive Review
• Direct Automatic Coronary Calcium Scoring in Cardiac and Chest CT
• CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions
• Learning Deformable Registration of Medical Images with Anatomical Constraints
• A coarse-to-fine framework for unsupervised multi-contrast MR image deformable registration with dual consistency constraint
• Medical image registration using unsupervised deep neural network: A scoping literature review
• Cortical surface registration using unsupervised learning
• mlVIRNET: Multilevel Variational Image Registration Network
• DeepReg: a deep learning toolkit for medical image registration
• Non-iterative Coarse-to-fine Transformer Networks for Joint Affine and Deformable Image Registration
• Learning the Effect of Registration Hyperparameters with HyperMorph
• Multimodal and multicontrast image fusion via deep generative models
• IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration
• AutoFuse: Automatic Fusion Networks for Deformable Medical Image Registration
• Enhancing Medical Image Registration via Appearance Adjustment Networks
• Anatomy-aware and acquisition-agnostic joint registration with SynthMorph
• Robust deformable image registration using cycle-consistent implicit representations
• Unsupervised domain adaptation based COVID-19 CT infection segmentation network
• Residual Aligner Network
• Non-iterative Coarse-to-fine Registration based on Single-pass Deep Cumulative Learning
• Generating Anthropomorphic Phantoms Using Fully Unsupervised Deformable Image Registration with Convolutional Neural Networks
• One shot PACS: Patient specific Anatomic Context and Shape prior aware recurrent registration-segmentation of longitudinal thoracic cone beam CTs
• AFFIRM: Affinity Fusion-based Framework for Iteratively Random Motion correction of multi-slice fetal brain MRI
• Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study
• Deep-Learning-based Fast and Accurate 3D CT Deformable Image Registration in Lung Cancer
• A Black-box Attack on Neural Networks Based on Swarm Evolutionary Algorithm
• NPBDREG: Uncertainty Assessment in Diffeomorphic Brain MRI Registration using a Non-parametric Bayesian Deep-Learning Based Approach
• CartiMorph: a framework for automated knee articular cartilage morphometrics
• CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration
• Contrastive Registration for Unsupervised Medical Image Segmentation
• Conditional Segmentation in Lieu of Image Registration
• SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI
• Shape constrained CNN for segmentation guided prediction of myocardial shape and pose parameters in cardiac MRI
• A Two-Step Deep Learning Method for 3DCT-2DUS Kidney Registration During Breathing
• FIRE: Unsupervised bi-directional inter-modality registration using deep networks
• Deep learning based registration using spatial gradients and noisy segmentation labels
• Recursive Deformable Image Registration Network with Mutual Attention
• MICDIR: Multi-scale Inverse-consistent Deformable Image Registration using UNetMSS with Self-Constructing Graph Latent
• Image-and-Spatial Transformer Networks for Structure-Guided Image Registration
• Recursive Refinement Network for Deformable Lung Registration between Exhale and Inhale CT Scans
• Few Labeled Atlases are Necessary for Deep-Learning-Based Segmentation
• Multi-Atlas Segmentation and Spatial Alignment of the Human Embryo in First Trimester 3D Ultrasound
• Deformable image registration with deep network priors: a study on longitudinal PET images
• ConKeD: Multiview contrastive descriptor learning for keypoint-based retinal image registration
• SITReg: Multi-resolution architecture for symmetric, inverse consistent, and topology preserving image registration
• Quasiconformal model with CNN features for large deformation image registration
• Unsupervised Deep-Learning Based Deformable Image Registration: A Bayesian Framework
• Deep Implicit Optimization enables Robust Learnable Features for Deformable Image Registration
• Semantic similarity metrics for learned image registration
• Test-Time Training for Deformable Multi-Scale Image Registration
• Unsupervised Deformable Medical Image Registration via Pyramidal Residual Deformation Fields Estimation
• Tackling the Problem of Large Deformations in Deep Learning Based Medical Image Registration Using Displacement Embeddings
• Attention for Image Registration (AiR): an unsupervised Transformer approach
• Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Images
• MDReg-Net: Multi-resolution diffeomorphic image registration using fully convolutional networks with deep self-supervision
• ABN: Anti-Blur Neural Networks for Multi-Stage Deformable Image Registration
• F3RNet: Full-Resolution Residual Registration Network for Deformable Image Registration
• A low-rank representation for unsupervised registration of medical images
• Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration
• KLDivNet: An unsupervised neural network for multi-modality image registration
• JSSR: A Joint Synthesis, Segmentation, and Registration System for 3D Multi-Modal Image Alignment of Large-scale Pathological CT Scans
• Cascaded Feature Warping Network for Unsupervised Medical Image Registration
• ProsRegNet: A Deep Learning Framework for Registration of MRI and Histopathology Images of the Prostate
• Eddeep: Fast eddy-current distortion correction for diffusion MRI with deep learning
• Weakly Supervised Registration of Prostate MRI and Histopathology Images
• Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation for 4D Dynamic Medical Images
• DeepDoseNet: A Deep Learning model for 3D Dose Prediction in Radiation Therapy
• Learning a Generative Motion Model from Image Sequences based on a Latent Motion Matrix
• DeepSim: Semantic similarity metrics for learned image registration
• CardioMorphNet: Cardiac Motion Prediction Using a Shape-Guided Bayesian Recurrent Deep Network
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation
• Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration
• Identifying partial mouse brain microscopy images from Allen reference atlas using a contrastively learned semantic space
• Temporal Periodic Image Registration with Implicit Neural Representations
• Regularized directional representations for medical image registration
• Joint cortical registration of geometry and function using semi-supervised learning
• Data-driven registration and modeling of brain deformation for image-guided neurosurgery
• Efficient Prealignment of CT Scans for Registration through a Bodypart Regressor
• AI for Calcium Scoring
• Artificial Intelligence in Minimally Invasive Interventional Treatment
• Multi-scale Neural ODEs for 3D Medical Image Registration
• D-Net: Siamese based Network with Mutual Attention for Volume Alignment