Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study

Abstract

Multi-modal images play a crucial role in comprehensive evaluations in medical image analysis providing complementary information for identifying clinically important biomarkers. However, in clinical practice, acquiring multiple modalities can be challenging due to reasons such as scan cost, limited scan time, and safety considerations. In this paper, we propose a model based on the latent diffusion model (LDM) that leverages switchable blocks for image-to-image translation in 3D medical images without patch cropping. The 3D LDM combined with conditioning using the target modality allows generating high-quality target modality in 3D overcoming the shortcoming of the missing out-of-slice information in 2D generation methods. The switchable block, noted as multiple switchable spatially adaptive normalization (MS-SPADE), dynamically transforms source latents to the desired style of the target latents to help with the diffusion process. The MS-SPADE block allows us to have one single model to tackle many translation tasks of one source modality to various targets removing the need for many translation models for different scenarios. Our model exhibited successful image synthesis across different source-target modality scenarios and surpassed other models in quantitative evaluations tested on multi-modal brain magnetic resonance imaging datasets of four different modalities and an independent IXI dataset. Our model demonstrated successful image synthesis across various modalities even allowing for one-to-many modality translations. Furthermore, it outperformed other one-to-one translation models in quantitative evaluations.

arXiv comment8 pages, 7 figures, WACV 2024 Accepted

References in corpus (8)

• Denoising Diffusion Probabilistic Models
• Neural Discrete Representation Learning
• Diffusion Models Beat GANs on Image Synthesis
• Generative Adversarial Network in Medical Imaging: A Review
• Diffusion Models in Vision: A Survey
• A Learned Representation For Artistic Style
• ResViT: Residual vision transformers for multi-modal medical image synthesis
• Brain tumor segmentation with missing modalities via latent multi-source correlation representation

Cited by in corpus (6)

• Deep learning for automated detection of breast cancer in deep ultraviolet fluorescence images with diffusion probabilistic model
• Tumor Synthesis conditioned on Radiomics
• Path and Bone-Contour Regularized Unpaired MRI-to-CT Translation
• Lesion-Aware Post-Training of Latent Diffusion Models for Synthesizing Diffusion MRI from CT Perfusion
• Pre to Post-Treatment Glioblastoma MRI Prediction using a Latent Diffusion Model
• Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis