DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy

Abstract

We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7 million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future.

Accepted for publication in NeuroImage, special issue "Brain Segmentation and Parcellation", 2017

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• How transferable are features in deep neural networks?
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations

Cited by in corpus (54)

• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• FastSurfer -- A fast and accurate deep learning based neuroimaging pipeline
• Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection
• End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reduction
• Segmentation of the Proximal Femur from MR Images using Deep Convolutional Neural Networks
• Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features
• Decoding and mapping task states of the human brain via deep learning
• FastSurferVINN: Building Resolution-Independence into Deep Learning Segmentation Methods -- A Solution for HighRes Brain MRI
• Bayesian convolutional neural network based MRI brain extraction on nonhuman primates
• Multiple Sclerosis Lesion Segmentation from Brain MRI via Fully Convolutional Neural Networks
• A Survey of Deep Learning for Scientific Discovery
• Hippocampus Segmentation on Epilepsy and Alzheimer's Disease Studies with Multiple Convolutional Neural Networks
• Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images
• End-to-end semantic segmentation of personalized deep brain structures for non-invasive brain stimulation
• AI for the prediction of early stages of Alzheimer's disease from neuroimaging biomarkers -- A narrative review of a growing field
• Deep learning in radiology: an overview of the concepts and a survey of the state of the art
• Recalibrating 3D ConvNets with Project & Excite
• Automated Claustrum Segmentation in Human Brain MRI Using Deep Learning
• Development of accurate human head models for personalized electromagnetic dosimetry using deep learning
• DARTS: DenseUnet-based Automatic Rapid Tool for brain Segmentation
• InfiNet: Fully Convolutional Networks for Infant Brain MRI Segmentation
• Auto-context Convolutional Neural Network (Auto-Net) for Brain Extraction in Magnetic Resonance Imaging
• Whole Brain Segmentation with Full Volume Neural Network
• Influence of segmentation accuracy in structural MR head scans on electric field computation for TMS and tES
• Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data
• Simultaneous lesion and neuroanatomy segmentation in Multiple Sclerosis using deep neural networks
• 3D Whole Brain Segmentation using Spatially Localized Atlas Network Tiles
• Research on Deep Learning-Based Semantic Segmentation Algorithms for Subcortical Brain Structures
• Spatially Localized Atlas Network Tiles Enables 3D Whole Brain Segmentation from Limited Data
• An End-to-end Approach to Semantic Segmentation with 3D CNN and Posterior-CRF in Medical Images
• Extreme Augmentation : Can deep learning based medical image segmentation be trained using a single manually delineated scan?
• Domain-agnostic segmentation of thalamic nuclei from joint structural and diffusion MRI
• Encoding Clinical Priori in 3D Convolutional Neural Networks for Prostate Cancer Detection in bpMRI
• Accurate Automatic Segmentation of Amygdala Subnuclei and Modeling of Uncertainty via Bayesian Fully Convolutional Neural Network
• Bayesian Spatial Binary Regression for Label Fusion in Structural Neuroimaging
• Isointense Infant Brain Segmentation with a Hyper-dense Connected Convolutional Neural Network
• Label merge-and-split: A graph-colouring approach for memory-efficient brain parcellation
• Reproducibility Evaluation of SLANT Whole Brain Segmentation Across Clinical Magnetic Resonance Imaging Protocols
• Enhancing Hierarchical Transformers for Whole Brain Segmentation with Intracranial Measurements Integration
• Feature-Fused Context-Encoding Network for Neuroanatomy Segmentation
• Bayesian QuickNAT: Model Uncertainty in Deep Whole-Brain Segmentation for Structure-wise Quality Control
• Pulse Sequence Resilient Fast Brain Segmentation
• Learnable Pooling in Graph Convolution Networks for Brain Surface Analysis
• ACEnet: Anatomical Context-Encoding Network for Neuroanatomy Segmentation
• Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems
• Why Calibration Error is Wrong Given Model Uncertainty: Using Posterior Predictive Checks with Deep Learning
• Beyond one-hot encoding? Journey into compact encoding for large multi-class segmentation
• Crossover-Net: Leveraging the Vertical-Horizontal Crossover Relation for Robust Segmentation
• Factorisation-based Image Labelling
• ErrorNet: Learning error representations from limited data to improve vascular segmentation
• Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review
• Does anatomical contextual information improve 3D U-Net based brain tumor segmentation?