An application of cascaded 3D fully convolutional networks for medical image segmentation

Abstract

Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of several anatomical structures (ranging from the large organs to thin vessels) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training class-specific models. To this end, we propose a two-stage, coarse-to-fine approach that will first use a 3D FCN to roughly define a candidate region, which will then be used as input to a second 3D FCN. This reduces the number of voxels the second FCN has to classify to ~10% and allows it to focus on more detailed segmentation of the organs and vessels. We utilize training and validation sets consisting of 331 clinical CT images and test our models on a completely unseen data collection acquired at a different hospital that includes 150 CT scans, targeting three anatomical organs (liver, spleen, and pancreas). In challenging organs such as the pancreas, our cascaded approach improves the mean Dice score from 68.5 to 82.2%, achieving the highest reported average score on this dataset. We compare with a 2D FCN method on a separate dataset of 240 CT scans with 18 classes and achieve a significantly higher performance in small organs and vessels. Furthermore, we explore fine-tuning our models to different datasets. Our experiments illustrate the promise and robustness of current 3D FCN based semantic segmentation of medical images, achieving state-of-the-art results. Our code and trained models are available for download: https://github.com/holgerroth/3Dunet_abdomen_cascade.

Preprint accepted for publication in Computerized Medical Imaging and Graphics. Substantial extension of arXiv:1704.06382; Corrected references to figure numbers in this version

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
• Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning
• Spatial Aggregation of Holistically-Nested Networks for Automated Pancreas Segmentation
• Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation
• A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans

Cited by in corpus (28)

• A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
• Abdominal multi-organ segmentation with organ-attention networks and statistical fusion
• Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets
• Deep and Statistical Learning in Biomedical Imaging: State of the Art in 3D MRI Brain Tumor Segmentation
• Pancreas segmentation with probabilistic map guided bi-directional recurrent UNet
• Deep Learning Based 3D Segmentation: A Survey
• Robust Iris Segmentation Based on Fully Convolutional Networks and Generative Adversarial Networks
• Globally Guided Progressive Fusion Network for 3D Pancreas Segmentation
• Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction
• Fully Convolutional Networks and Generative Adversarial Networks Applied to Sclera Segmentation
• Precise Estimation of Renal Vascular Dominant Regions Using Spatially Aware Fully Convolutional Networks, Tensor-Cut and Voronoi Diagrams
• Deep Learning for Pancreas Segmentation: a Systematic Review
• Multi-level Context Gating of Embedded Collective Knowledge for Medical Image Segmentation
• AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• CT organ segmentation using GPU data augmentation, unsupervised labels and IOU loss
• Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction
• A Model-Driven Stack-Based Fully Convolutional Network for Pancreas Segmentation
• Deep Learning in Multi-organ Segmentation
• Imperfect Segmentation Labels: How Much Do They Matter?
• Vertebrae Detection and Localization in CT with Two-Stage CNNs and Dense Annotations
• COVID-19 Infection Segmentation from Chest CT Images Based on Scale Uncertainty
• Ensemble Methods for Multi-Organ Segmentation in CT Series
• Synthetic vascular structure generation for unsupervised pre-training in CTA segmentation tasks
• Immunofluorescence Capillary Imaging Segmentation: Cases Study
• OARnet: Automated organs-at-risk delineation in Head and Neck CT images
• Deep Mouse: An End-to-end Auto-context Refinement Framework for Brain Ventricle and Body Segmentation in Embryonic Mice Ultrasound Volumes
• Automatic Segmentation, Localization, and Identification of Vertebrae in 3D CT Images Using Cascaded Convolutional Neural Networks