Scalable multimodal convolutional networks for brain tumour segmentation

Abstract

Brain tumour segmentation plays a key role in computer-assisted surgery. Deep neural networks have increased the accuracy of automatic segmentation significantly, however these models tend to generalise poorly to different imaging modalities than those for which they have been designed, thereby limiting their applications. For example, a network architecture initially designed for brain parcellation of monomodal T1 MRI can not be easily translated into an efficient tumour segmentation network that jointly utilises T1, T1c, Flair and T2 MRI. To tackle this, we propose a novel scalable multimodal deep learning architecture using new nested structures that explicitly leverage deep features within or across modalities. This aims at making the early layers of the architecture structured and sparse so that the final architecture becomes scalable to the number of modalities. We evaluate the scalable architecture for brain tumour segmentation and give evidence of its regularisation effect compared to the conventional concatenation approach.

Paper accepted at MICCAI 2017

References in corpus (4)

• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Brain Tumor Segmentation with Deep Neural Networks
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups

Cited by in corpus (12)

• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
• DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation
• Disentangled Representation Learning in Cardiac Image Analysis
• Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks
• Generalized Wasserstein Dice Score, Distributionally Robust Deep Learning, and Ranger for brain tumor segmentation: BraTS 2020 challenge
• Modality-Pairing Learning for Brain Tumor Segmentation
• Unpaired Multi-modal Segmentation via Knowledge Distillation
• Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation
• A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers
• Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion