A deep learning model integrating FCNNs and CRFs for brain tumor segmentation

Abstract

Accurate and reliable brain tumor segmentation is a critical component in cancer diagnosis, treatment planning, and treatment outcome evaluation. Build upon successful deep learning techniques, a novel brain tumor segmentation method is developed by integrating fully convolutional neural networks (FCNNs) and Conditional Random Fields (CRFs) in a unified framework to obtain segmentation results with appearance and spatial consistency. We train a deep learning based segmentation model using 2D image patches and image slices in following steps: 1) training FCNNs using image patches; 2) training CRFs as Recurrent Neural Networks (CRF-RNN) using image slices with parameters of FCNNs fixed; and 3) fine-tuning the FCNNs and the CRF-RNN using image slices. Particularly, we train 3 segmentation models using 2D image patches and slices obtained in axial, coronal and sagittal views respectively, and combine them to segment brain tumors using a voting based fusion strategy. Our method could segment brain images slice-by-slice, much faster than those based on image patches. We have evaluated our method based on imaging data provided by the Multimodal Brain Tumor Image Segmentation Challenge (BRATS) 2013, BRATS 2015 and BRATS 2016. The experimental results have demonstrated that our method could build a segmentation model with Flair, T1c, and T2 scans and achieve competitive performance as those built with Flair, T1, T1c, and T2 scans.

arXiv commentThis version was accepted in the journal Medical Image Analysis

References in corpus (10)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Brain Tumor Segmentation with Deep Neural Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Conditional Random Fields as Recurrent Neural Networks
• Automatic segmentation of MR brain images with a convolutional neural network
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• DALSA: Domain Adaptation for Supervised Learning From Sparsely Annotated MR Images
• 3-D Convolutional Neural Networks for Glioblastoma Segmentation

Cited by in corpus (33)

• A review: Deep learning for medical image segmentation using multi-modality fusion
• Deep Learning Enables Automatic Detection and Segmentation of Brain Metastases on Multi-Sequence MRI
• One-pass Multi-task Networks with Cross-task Guided Attention for Brain Tumor Segmentation
• Deep and Statistical Learning in Biomedical Imaging: State of the Art in 3D MRI Brain Tumor Segmentation
• Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks
• Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks
• PA-ResSeg: A Phase Attention Residual Network for Liver Tumor Segmentation from Multi-phase CT Images
• MyoPS-Net: Myocardial Pathology Segmentation with Flexible Combination of Multi-Sequence CMR Images
• A Transformer-based Generative Adversarial Network for Brain Tumor Segmentation
• Brain tumor segmentation with missing modalities via latent multi-source correlation representation
• Adaptive feature recombination and recalibration for semantic segmentation: application to brain tumor segmentation in MRI
• Fast meningioma segmentation in T1-weighted MRI volumes using a lightweight 3D deep learning architecture
• Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation
• Unveiling the frontiers of deep learning: innovations shaping diverse domains
• Trichomonas Vaginalis Segmentation in Microscope Images
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model
• Deep Learning in Computer-Aided Diagnosis and Treatment of Tumors: A Survey
• A Feature Transfer Enabled Multi-Task Deep Learning Model on Medical Imaging
• Fully-automatic segmentation of kidneys in clinical ultrasound images using a boundary distance regression network
• MRI Pulse Sequence Integration for Deep-Learning Based Brain Metastasis Segmentation
• A Volumetric Convolutional Neural Network for Brain Tumor Segmentation
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding
• Towards Automatic 3D Shape Instantiation for Deployed Stent Grafts: 2D Multiple-class and Class-imbalance Marker Segmentation with Equally-weighted Focal U-Net
• Artificial Intelligence in Tumor Subregion Analysis Based on Medical Imaging: A Review
• HIVE-Net: Centerline-Aware HIerarchical View-Ensemble Convolutional Network for Mitochondria Segmentation in EM Images
• ACEnet: Anatomical Context-Encoding Network for Neuroanatomy Segmentation
• A Tri-attention Fusion Guided Multi-modal Segmentation Network
• Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion
• Convolution Neural Networks for Semantic Segmentation: Application to Small Datasets of Biomedical Images
• Convolutional herbal prescription building method from multi-scale facial features
• A Multi-View Dynamic Fusion Framework: How to Improve the Multimodal Brain Tumor Segmentation from Multi-Views?
• 3D Medical Multi-modal Segmentation Network Guided by Multi-source Correlation Constraint