Cross-Modality Deep Feature Learning for Brain Tumor Segmentation

Abstract

Recent advances in machine learning and prevalence of digital medical images have opened up an opportunity to address the challenging brain tumor segmentation (BTS) task by using deep convolutional neural networks. However, different from the RGB image data that are very widespread, the medical image data used in brain tumor segmentation are relatively scarce in terms of the data scale but contain the richer information in terms of the modality property. To this end, this paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. The proposed cross-modality deep feature learning framework consists of two learning processes: the cross-modality feature transition (CMFT) process and the cross-modality feature fusion (CMFF) process, which aims at learning rich feature representations by transiting knowledge across different modality data and fusing knowledge from different modality data, respectively. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance when compared with the baseline methods and state-of-the-art methods.

arXiv commentpublished on Pattern Recognition 2021

References in corpus (11)

• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
• Perceptual Adversarial Networks for Image-to-Image Transformation
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• CBAM: Convolutional Block Attention Module
• Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks
• Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation
• Scalable multimodal convolutional networks for brain tumour segmentation
• Conditional Adversarial Network for Semantic Segmentation of Brain Tumor
• Global Planar Convolutions for improved context aggregation in Brain Tumor Segmentation

Cited by in corpus (12)

• Deep Industrial Image Anomaly Detection: A Survey
• Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities
• CPP-Net: Context-aware Polygon Proposal Network for Nucleus Segmentation
• Deep and Statistical Learning in Biomedical Imaging: State of the Art in 3D MRI Brain Tumor Segmentation
• A Novel Framework for Brain Tumor Detection Based on Convolutional Variational Generative Models
• Geodesic Paths for Image Segmentation with Implicit Region-based Homogeneity Enhancement
• MSCoTDet: Language-driven Multi-modal Fusion for Improved Multispectral Pedestrian Detection
• 3D Brain and Heart Volume Generative Models: A Survey
• HyCTAS: Multi-Objective Hybrid Convolution-Transformer Architecture Search for Real-Time Image Segmentation
• sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging
• A Tri-attention Fusion Guided Multi-modal Segmentation Network
• Quasi-multimodal-based pathophysiological feature learning for retinal disease diagnosis