TransMorph: Transformer for unsupervised medical image registration

Abstract

In the last decade, convolutional neural networks (ConvNets) have been a major focus of research in medical image analysis. However, the performances of ConvNets may be limited by a lack of explicit consideration of the long-range spatial relationships in an image. Recently Vision Transformer architectures have been proposed to address the shortcomings of ConvNets and have produced state-of-the-art performances in many medical imaging applications. Transformers may be a strong candidate for image registration because their substantially larger receptive field enables a more precise comprehension of the spatial correspondence between moving and fixed images. Here, we present TransMorph, a hybrid Transformer-ConvNet model for volumetric medical image registration. This paper also presents diffeomorphic and Bayesian variants of TransMorph: the diffeomorphic variants ensure the topology-preserving deformations, and the Bayesian variant produces a well-calibrated registration uncertainty estimate. We extensively validated the proposed models using 3D medical images from three applications: inter-patient and atlas-to-patient brain MRI registration and phantom-to-CT registration. The proposed models are evaluated in comparison to a variety of existing registration methods and Transformer architectures. Qualitative and quantitative results demonstrate that the proposed Transformer-based model leads to a substantial performance improvement over the baseline methods, confirming the effectiveness of Transformers for medical image registration.

Accepted to Medical Image Analysis ((c) MedIA). Code available at https://bit.ly/37eJS6N | This version: Several typographical errors were fixed

References in corpus (6)

• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• How Do Vision Transformers Work?
• Qualitatively characterizing neural network optimization problems
• ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration

Cited by in corpus (30)

• Transformers in Healthcare: A Survey
• Recent Progress in Transformer-based Medical Image Analysis
• ModeT: Learning Deformable Image Registration via Motion Decomposition Transformer
• Non-iterative Coarse-to-fine Transformer Networks for Joint Affine and Deformable Image Registration
• Artificial General Intelligence for Medical Imaging Analysis
• AutoFuse: Automatic Fusion Networks for Deformable Medical Image Registration
• SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images
• UTSRMorph: A Unified Transformer and Superresolution Network for Unsupervised Medical Image Registration
• Residual Aligner Network
• Multi-source Domain Adaptation for Panoramic Semantic Segmentation
• SF2Former: Amyotrophic Lateral Sclerosis Identification From Multi-center MRI Data Using Spatial and Frequency Fusion Transformer
• Attention-aware non-rigid image registration for accelerated MR imaging
• CartiMorph: a framework for automated knee articular cartilage morphometrics
• Generative Aging of Brain Images with Diffeomorphic Registration
• Recursive Deformable Image Registration Network with Mutual Attention
• A Two-Step Deep Learning Method for 3DCT-2DUS Kidney Registration During Breathing
• Mamba? Catch The Hype Or Rethink What Really Helps for Image Registration
• Deep Implicit Optimization enables Robust Learnable Features for Deformable Image Registration
• Tumor aware recurrent inter-patient deformable image registration of computed tomography scans with lung cancer
• ModeTv2: GPU-accelerated Motion Decomposition Transformer for Pairwise Optimization in Medical Image Registration
• MRUCT: Mixed Reality Assistance for Acupuncture Guided by Ultrasonic Computed Tomography
• Gaussian Primitives for Deformable Image Registration
• Sparse Mixture-of-Experts for Non-Uniform Noise Reduction in MRI Images
• FetDTIAlign: A Deep Learning Framework for Affine and Deformable Registration of Fetal Brain dMRI
• Synthesizing Individualized Aging Brains in Health and Disease with Generative Models and Parallel Transport
• RFR-WWANet: Weighted Window Attention-Based Recovery Feature Resolution Network for Unsupervised Image Registration
• Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration
• Regional Deep Atrophy: a Self-Supervised Learning Method to Automatically Identify Regions Associated With Alzheimer's Disease Progression From Longitudinal MRI
• TransAnaNet: Transformer-based Anatomy Change Prediction Network for Head and Neck Cancer Patient Radiotherapy
• Data-driven registration and modeling of brain deformation for image-guided neurosurgery