Whole-Body Image-to-Image Translation for a Virtual Scanner in a Healthcare Digital Twin

Abstract

Generating positron emission tomography (PET) images from computed tomography (CT) scans via deep learning offers a promising pathway to reduce radiation exposure and costs associated with PET imaging, improving patient care and accessibility to functional imaging. Whole-body image translation presents challenges due to anatomical heterogeneity, often limiting generalized models. We propose a framework that segments whole-body CT images into four regions-head, trunk, arms, and legs-and uses district-specific Generative Adversarial Networks (GANs) for tailored CT-to-PET translation. Synthetic PET images from each region are stitched together to reconstruct the whole-body scan. Comparisons with a baseline non-segmented GAN and experiments with Pix2Pix and CycleGAN architectures tested paired and unpaired scenarios. Quantitative evaluations at district, whole-body, and lesion levels demonstrated significant improvements with our district-specific GANs. Pix2Pix yielded superior metrics, ensuring precise, high-quality image synthesis. By addressing anatomical heterogeneity, this approach achieves state-of-the-art results in whole-body CT-to-PET translation. This methodology supports healthcare Digital Twins by enabling accurate virtual PET scans from CT data, creating virtual imaging representations to monitor, predict, and optimize health outcomes.

References in corpus (7)

• A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications
• A Deep Learning Approach for Virtual Contrast Enhancement in Contrast Enhanced Spectral Mammography
• A Deep Learning Approach for Overall Survival Prediction in Lung Cancer with Missing Values
• Multimodal Explainability via Latent Shift applied to COVID-19 stratification
• Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes
• A graph neural network-based model with Out-of-Distribution Robustness for enhancing Antiretroviral Therapy Outcome Prediction for HIV-1
• Multi-Dataset Multi-Task Learning for COVID-19 Prognosis

Cited by in corpus (1)

• Virtual Scanning for NSCLC Histology: Investigating the Discriminatory Power of Synthetic PET