Brain tumor segmentation with missing modalities via latent multi-source correlation representation

Abstract

Multimodal MR images can provide complementary information for accurate brain tumor segmentation. However, it's common to have missing imaging modalities in clinical practice. Since there exists a strong correlation between multi modalities, a novel correlation representation block is proposed to specially discover the latent multi-source correlation. Thanks to the obtained correlation representation, the segmentation becomes more robust in the case of missing modalities. The model parameter estimation module first maps the individual representation produced by each encoder to obtain independent parameters, then, under these parameters, the correlation expression module transforms all the individual representations to form a latent multi-source correlation representation. Finally, the correlation representations across modalities are fused via the attention mechanism into a shared representation to emphasize the most important features for segmentation. We evaluate our model on BraTS 2018 datasets, it outperforms the current state-of-the-art method and produces robust results when one or more modalities are missing.

arXiv comment10 pages, 6 figures, accepted by MICCAI 2020. arXiv admin note: text overlap with arXiv:2102.03111

References in corpus (5)

• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Brain Tumor Segmentation with Deep Neural Networks
• A deep learning model integrating FCNNs and CRFs for brain tumor segmentation
• A review: Deep learning for medical image segmentation using multi-modality fusion
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks

Cited by in corpus (2)

• An automatic COVID-19 CT segmentation network using spatial and channel attention mechanism
• Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study