A Two-Step Deep Learning Method for 3DCT-2DUS Kidney Registration During Breathing

Abstract

This work proposed a novel deep registration pipeline for 3D CT and 2D U/S kidney scans of free breathing, which consists of a feature network, and a 3D-2D CNN-based registration network. The feature network has handcraft texture feature layers to reduce the semantic gap. The registration network is encoder-decoder structure with loss of feature-image-motion (FIM), which enables hierarchical regression at decoder layers and avoids multiple network concatenation. It was first pretrained with retrospective datasets cum training data generation strategy, then adapted to specific patient data under unsupervised one-cycle transfer learning in onsite application. The experiment was on 132 U/S sequences, 39 multiple phase CT and 210 public single phase CT images, and 25 pairs of CT and U/S sequences. It resulted in mean contour distance (MCD) of 0.94 mm between kidneys on CT and U/S images and MCD of 1.15 mm on CT and reference CT images. For datasets with small transformations, it resulted in MCD of 0.82 and 1.02 mm respectively. For large transformations, it resulted in MCD of 1.10 and 1.28 mm respectively. This work addressed difficulties in 3DCT-2DUS kidney registration during free breathing via novel network structures and training strategy.

arXiv comment16 pages, 8 figures, 10 tables

References in corpus (10)

• VoxelMorph: A Learning Framework for Deformable Medical Image Registration
• A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration
• TransMorph: Transformer for unsupervised medical image registration
• Deep Learning in Medical Image Registration: A Survey
• Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration
• Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration
• Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network
• ssEMnet: Serial-section Electron Microscopy Image Registration using a Spatial Transformer Network with Learned Features
• Adversarial Deformation Regularization for Training Image Registration Neural Networks
• Real-Time 2D-3D Deformable Registration with Deep Learning and Application to Lung Radiotherapy Targeting