IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration

Abstract

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a 3D organ mesh for a single-viewpoint 2D projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.

References in corpus (2)

• Real-time markerless tumour tracking with patient-specific deep learning using a personalized data generation strategy: Proof of concept by phantom study
• Image-to-Graph Convolutional Network for Deformable Shape Reconstruction from a Single Projection Image

Cited by in corpus (3)

• A Review on Organ Deformation Modeling Approaches for Reliable Surgical Navigation using Augmented Reality
• Enhancing medical vision-language contrastive learning via inter-matching relation modelling
• Graph Convolutional Neural Networks for Automated Echocardiography View Recognition: A Holistic Approach