Brain Tumor Segmentation with Deep Neural Networks

Abstract

In this paper, we present a fully automatic brain tumor segmentation method based on Deep Neural Networks (DNNs). The proposed networks are tailored to glioblastomas (both low and high grade) pictured in MR images. By their very nature, these tumors can appear anywhere in the brain and have almost any kind of shape, size, and contrast. These reasons motivate our exploration of a machine learning solution that exploits a flexible, high capacity DNN while being extremely efficient. Here, we give a description of different model choices that we've found to be necessary for obtaining competitive performance. We explore in particular different architectures based on Convolutional Neural Networks (CNN), i.e. DNNs specifically adapted to image data. We present a novel CNN architecture which differs from those traditionally used in computer vision. Our CNN exploits both local features as well as more global contextual features simultaneously. Also, different from most traditional uses of CNNs, our networks use a final layer that is a convolutional implementation of a fully connected layer which allows a 40 fold speed up. We also describe a 2-phase training procedure that allows us to tackle difficulties related to the imbalance of tumor labels. Finally, we explore a cascade architecture in which the output of a basic CNN is treated as an additional source of information for a subsequent CNN. Results reported on the 2013 BRATS test dataset reveal that our architecture improves over the currently published state-of-the-art while being over 30 times faster.

References in corpus (4)

• Brain Tumor Segmentation with Deep Neural Networks
• Fully Convolutional Networks for Semantic Segmentation
• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• A Generalized Mean Field Algorithm for Variational Inference in Exponential Families

Cited by in corpus (216)

• A Survey on Deep Learning in Medical Image Analysis
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Brain Tumor Segmentation with Deep Neural Networks
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• A systematic study of the class imbalance problem in convolutional neural networks
• Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
• An overview of deep learning in medical imaging focusing on MRI
• Medical Image Analysis using Convolutional Neural Networks: A Review
• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• Applications of Deep Learning and Reinforcement Learning to Biological Data
• A deep learning model integrating FCNNs and CRFs for brain tumor segmentation
• Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease
• Automated brain extraction of multi-sequence MRI using artificial neural networks
• FastSurfer -- A fast and accurate deep learning based neuroimaging pipeline
• A review: Deep learning for medical image segmentation using multi-modality fusion
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• A Deep Learning Approach for Brain Tumor Classification and Segmentation Using a Multiscale Convolutional Neural Network
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
• Boundary loss for highly unbalanced segmentation
• DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
• Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
• 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study
• Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review
• Centralized Feature Pyramid for Object Detection
• Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
• Loss Functions and Metrics in Deep Learning
• Machine Learning Techniques for Biomedical Image Segmentation: An Overview of Technical Aspects and Introduction to State-of-Art Applications
• Segmentation of Glioma Tumors in Brain Using Deep Convolutional Neural Network
• Abdominal multi-organ segmentation with organ-attention networks and statistical fusion
• Fully Convolutional Network Ensembles for White Matter Hyperintensities Segmentation in MR Images
• DeepSeg: Deep Neural Network Framework for Automatic Brain Tumor Segmentation using Magnetic Resonance FLAIR Images
• Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities
• One-pass Multi-task Networks with Cross-task Guided Attention for Brain Tumor Segmentation
• Radiological images and machine learning: trends, perspectives, and prospects
• Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis
• Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection
• Attention Enriched Deep Learning Model for Breast Tumor Segmentation in Ultrasound Images
• Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation
• Multi-branch Convolutional Neural Network for Multiple Sclerosis Lesion Segmentation
• Uncertainty and Interpretability in Convolutional Neural Networks for Semantic Segmentation of Colorectal Polyps
• Deep Attentive Features for Prostate Segmentation in 3D Transrectal Ultrasound
• Automatic Brain Tumor Segmentation using Convolutional Neural Networks with Test-Time Augmentation
• CNN-based Segmentation of Medical Imaging Data
• Multi-Scale Supervised 3D U-Net for Kidneys and Kidney Tumor Segmentation
• Handwritten Isolated Bangla Compound Character Recognition: a new benchmark using a novel deep learning approach
• Attention-Guided Version of 2D UNet for Automatic Brain Tumor Segmentation
• A Deep Learning Framework for Single-Sided Sound Speed Inversion in Medical Ultrasound
• Cascaded V-Net using ROI masks for brain tumor segmentation
• The Importance of Skip Connections in Biomedical Image Segmentation
• Crossbar-Net: A Novel Convolutional Network for Kidney Tumor Segmentation in CT Images
• Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation
• HoloStain: Holographic virtual staining of individual biological cells
• Decoding and mapping task states of the human brain via deep learning
• Pose-Aware Instance Segmentation Framework from Cone Beam CT Images for Tooth Segmentation
• Dilated Convolutional Neural Networks for Cardiovascular MR Segmentation in Congenital Heart Disease
• Shape and Margin-Aware Lung Nodule Classification in Low-dose CT Images via Soft Activation Mapping
• Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation
• Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks
• Deep and Statistical Learning in Biomedical Imaging: State of the Art in 3D MRI Brain Tumor Segmentation
• Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks
• POBA-GA: Perturbation Optimized Black-Box Adversarial Attacks via Genetic Algorithm
• Causal Intervention for Weakly-Supervised Semantic Segmentation
• Extraction of Skin Lesions from Non-Dermoscopic Images Using Deep Learning
• Disentangle, align and fuse for multimodal and semi-supervised image segmentation
• Integrating uncertainty in deep neural networks for MRI based stroke analysis
• Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval
• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• Scalable multimodal convolutional networks for brain tumour segmentation
• Artificial Intelligence in Glioma Imaging: Challenges and Advances
• Circuit designs for superconducting optoelectronic loop neurons
• Brain segmentation based on multi-atlas guided 3D fully convolutional network ensembles
• Computer-Aided Diagnosis of Label-Free 3-D Optical Coherence Microscopy Images of Human Cervical Tissue
• Deep Learning Convolutional Networks for Multiphoton Microscopy Vasculature Segmentation
• Deep Radiomics for Brain Tumor Detection and Classification from Multi-Sequence MRI
• Brain tumor segmentation with missing modalities via latent multi-source correlation representation
• Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images
• Multimodal and multicontrast image fusion via deep generative models
• Multiparametric Deep Learning Tissue Signatures for a Radiological Biomarker of Breast Cancer: Preliminary Results
• A Deep Learning Approach for Pose Estimation from Volumetric OCT Data
• Deeply Self-Supervised Contour Embedded Neural Network Applied to Liver Segmentation
• Sequential 3D U-Nets for Biologically-Informed Brain Tumor Segmentation
• Deep learning in radiology: an overview of the concepts and a survey of the state of the art
• Deep Neural Network with l2-norm Unit for Brain Lesions Detection
• TBI Contusion Segmentation from MRI using Convolutional Neural Networks
• Hough-CNN: Deep Learning for Segmentation of Deep Brain Regions in MRI and Ultrasound
• Conditional Generation of Medical Images via Disentangled Adversarial Inference
• Liver Segmentation in Abdominal CT Images via Auto-Context Neural Network and Self-Supervised Contour Attention
• CASED: Curriculum Adaptive Sampling for Extreme Data Imbalance
• ASMCNN: An Efficient Brain Extraction Using Active Shape Model and Convolutional Neural Networks
• A Survey and Analysis on Automated Glioma Brain Tumor Segmentation and Overall Patient Survival Prediction
• Precise Estimation of Renal Vascular Dominant Regions Using Spatially Aware Fully Convolutional Networks, Tensor-Cut and Voronoi Diagrams
• Thickened 2D Networks for Efficient 3D Medical Image Segmentation
• Development of accurate human head models for personalized electromagnetic dosimetry using deep learning
• Brain Tumor Segmentation Based on Refined Fully Convolutional Neural Networks with A Hierarchical Dice Loss
• Joint Sequence Learning and Cross-Modality Convolution for 3D Biomedical Segmentation
• Fast meningioma segmentation in T1-weighted MRI volumes using a lightweight 3D deep learning architecture
• A comparative study of 2D image segmentation algorithms for traumatic brain lesions using CT data from the ProTECTIII multicenter clinical trial
• Adversarial Time-to-Event Modeling
• Adversarial training and dilated convolutions for brain MRI segmentation
• Synthesis of Brain Tumor MR Images for Learning Data Augmentation
• External Attention Assisted Multi-Phase Splenic Vascular Injury Segmentation with Limited Data
• Brain tumour segmentation using a triplanar ensemble of U-Nets
• Brain MRI Tumor Segmentation with Adversarial Networks
• Deep Learning in Bioinformatics
• MRI Tumor Segmentation with Densely Connected 3D CNN
• A Survey on Deep Learning-based Non-Invasive Brain Signals:Recent Advances and New Frontiers
• Learning Shape Representation on Sparse Point Clouds for Volumetric Image Segmentation
• Towards fully automated deep-learning-based brain tumor segmentation: is brain extraction still necessary?
• M2Net: Multi-modal Multi-channel Network for Overall Survival Time Prediction of Brain Tumor Patients
• Trichomonas Vaginalis Segmentation in Microscope Images
• Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach
• Convolutional 3D to 2D Patch Conversion for Pixel-wise Glioma Segmentation in MRI Scans
• Bottleneck Supervised U-Net for Pixel-wise Liver and Tumor Segmentation
• Few-shot learning for COVID-19 Chest X-Ray Classification with Imbalanced Data: An Inter vs. Intra Domain Study
• Detection and Tracking of Liquids with Fully Convolutional Networks
• Non-inferiority of Deep Learning Acute Ischemic Stroke Segmentation on Non-Contrast CT Compared to Expert Neuroradiologists
• U-Det: A Modified U-Net architecture with bidirectional feature network for lung nodule segmentation
• Dropout: Explicit Forms and Capacity Control
• Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps
• Dilated Inception U-Net (DIU-Net) for Brain Tumor Segmentation
• Interactive Medical Image Segmentation via Point-Based Interaction and Sequential Patch Learning
• MMMNA-Net for Overall Survival Time Prediction of Brain Tumor Patients
• Multimodal MRI brain tumor segmentation using random forests with features learned from fully convolutional neural network
• Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model
• DINs: Deep Interactive Networks for Neurofibroma Segmentation in Neurofibromatosis Type 1 on Whole-Body MRI
• Improved Workflow for Unsupervised Multiphase Image Segmentation
• Artificial Intelligence-Based Image Classification for Diagnosis of Skin Cancer: Challenges and Opportunities
• PanoNet: Real-time Panoptic Segmentation through Position-Sensitive Feature Embedding
• A Combined Deep Learning-Gradient Boosting Machine Framework for Fluid Intelligence Prediction
• A CADe System for Gliomas in Brain MRI using Convolutional Neural Networks
• 3D Dilated Multi-Fiber Network for Real-time Brain Tumor Segmentation in MRI
• ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities
• Data Augmentation For Medical MR Image Using Generative Adversarial Networks
• Unsupervised domain adaptation in brain lesion segmentation with adversarial networks
• Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation
• A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation
• Superconducting Optoelectronic Neurons I: General Principles
• Automatic Cell Counting in Flourescent Microscopy Using Deep Learning
• Fast and Accurate 3D Medical Image Segmentation with Data-swapping Method
• Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation
• Segmenting Brain Tumors with Symmetry
• Elastic Boundary Projection for 3D Medical Image Segmentation
• Adaptive Early-Learning Correction for Segmentation from Noisy Annotations
• Densely Connected Recurrent Residual (Dense R2UNet) Convolutional Neural Network for Segmentation of Lung CT Images
• Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data
• Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
• A Feature Transfer Enabled Multi-Task Deep Learning Model on Medical Imaging
• Early Detection of Retinopathy of Prematurity (ROP) in Retinal Fundus Images Via Convolutional Neural Networks
• Multi-step Cascaded Networks for Brain Tumor Segmentation
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Enhanced 3D Brain Tumor Segmentation Using Assorted Precision Training
• Persian Signature Verification using Fully Convolutional Networks
• Colorectal Cancer Outcome Prediction from H&E Whole Slide Images using Machine Learning and Automatically Inferred Phenotype Profiles
• Deep Distance Map Regression Network with Shape-aware Loss for Imbalanced Medical Image Segmentation
• MMFNet: A Multi-modality MRI Fusion Network for Segmentation of Nasopharyngeal Carcinoma
• Cross-Modality Brain Tumor Segmentation via Bidirectional Global-to-Local Unsupervised Domain Adaptation
• Multiclass Spinal Cord Tumor Segmentation on MRI with Deep Learning
• A multi-path 2.5 dimensional convolutional neural network system for segmenting stroke lesions in brain MRI images
• Accurate Lung Nodules Segmentation with Detailed Representation Transfer and Soft Mask Supervision
• Machine Learning Prediction of COVID-19 Severity Levels From Salivaomics Data
• Upper Bounds on the Generalization Error of Private Algorithms for Discrete Data
• CU-Net: Cascaded U-Net with Loss Weighted Sampling for Brain Tumor Segmentation
• Weakly-Supervised Universal Lesion Segmentation with Regional Level Set Loss
• Triplet Contrastive Learning for Brain Tumor Classification
• A Volumetric Convolutional Neural Network for Brain Tumor Segmentation
• 3D Convolutional Neural Networks for Brain Tumor Segmentation: A Comparison of Multi-resolution Architectures
• Brain Tumor Segmentation Network Using Attention-based Fusion and Spatial Relationship Constraint
• Spectral Graph Convolutions for Population-based Disease Prediction
• Exploiting bilateral symmetry in brain lesion segmentation
• A Multi-task Contextual Atrous Residual Network for Brain Tumor Detection & Segmentation
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding
• TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors
• Spinal Metastases Segmentation in MR Imaging using Deep Convolutional Neural Networks
• Learning More with Less: Conditional PGGAN-based Data Augmentation for Brain Metastases Detection Using Highly-Rough Annotation on MR Images
• Lesion Conditional Image Generation for Improved Segmentation of Intracranial Hemorrhage from CT Images
• Using Persistent Homology Topological Features to Characterize Medical Images: Case Studies on Lung and Brain Cancers
• HeMIS: Hetero-Modal Image Segmentation
• Multimodal 3D Brain Tumor Segmentation with Adversarial Training and Conditional Random Field
• Deep Learning Based Segmentation of Various Brain Lesions for Radiosurgery
• Fully Automated and Standardized Segmentation of Adipose Tissue Compartments by Deep Learning in Three-dimensional Whole-body MRI of Epidemiological Cohort Studies
• Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling
• Bengali Handwritten Grapheme Classification: Deep Learning Approach
• Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning
• Pixel Relationships-based Regularizer for Retinal Vessel Image Segmentation
• Label Super Resolution with Inter-Instance Loss
• Computer-Assisted Analysis of Biomedical Images
• Phase Collaborative Network for Two-Phase Medical Image Segmentation
• Unsupervised Data Selection for Supervised Learning
• Artificial Intelligence in Tumor Subregion Analysis Based on Medical Imaging: A Review
• Max-Fusion U-Net for Multi-Modal Pathology Segmentation with Attention and Dynamic Resampling
• Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
• Semi-supervised Learning using Denoising Autoencoders for Brain Lesion Detection and Segmentation
• Soft labeling by Distilling Anatomical knowledge for Improved MS Lesion Segmentation
• A lateral semicircular canal segmentation based geometric calibration for human temporal bone CT Image
• Crossover-Net: Leveraging the Vertical-Horizontal Crossover Relation for Robust Segmentation
• 3D Medical Multi-modal Segmentation Network Guided by Multi-source Correlation Constraint
• Segmentation of brain tumor on magnetic resonance imaging using a convolutional architecture
• Brain Tumor Segmentation on MRI with Missing Modalities
• A Multi-View Dynamic Fusion Framework: How to Improve the Multimodal Brain Tumor Segmentation from Multi-Views?
• Multi-Scale Gradual Integration CNN for False Positive Reduction in Pulmonary Nodule Detection
• Statistical inference of the inter-sample Dice distribution for discriminative CNN brain lesion segmentation models
• Perceiving and Reasoning About Liquids Using Fully Convolutional Networks
• Modelling brain lesion volume in patches with CNN-based Poisson Regression
• Self-Regulation for Semantic Segmentation
• CASPIANET++: A Multidimensional Channel-Spatial Asymmetric Attention Network with Noisy Student Curriculum Learning Paradigm for Brain Tumor Segmentation
• Pathology-Aware Generative Adversarial Networks for Medical Image Augmentation
• Why Calibration Error is Wrong Given Model Uncertainty: Using Posterior Predictive Checks with Deep Learning
• Deep learning and whole-brain networks for biomarker discovery: modeling the dynamics of brain fluctuations in resting-state and cognitive tasks
• AutoAtlas: Neural Network for 3D Unsupervised Partitioning and Representation Learning
• Did Evolution get it right? An evaluation of Near-Infrared imaging in semantic scene segmentation using deep learning
• Tissue characterization based on the analysis on i3DUS data for diagnosis support in neurosurgery
• 3D U-Net for Segmentation of Plant Root MRI Images in Super-Resolution
• Integrated Model, Batch and Domain Parallelism in Training Neural Networks