Hierarchical 3D fully convolutional networks for multi-organ segmentation

Abstract

Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of full volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of seven abdominal structures (artery, vein, liver, spleen, stomach, gallbladder, and pancreas) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training organ-specific models. To this end, we propose a two-stage, coarse-to-fine approach that trains an FCN model to roughly delineate the organs of interest in the first stage (seeing $\sim$40% of the voxels within a simple, automatically generated binary mask of the patient's body). We then use these predictions of the first-stage FCN to define a candidate region that will be used to train a second FCN. This step reduces the number of voxels the FCN has to classify to $\sim$10% while maintaining a recall high of $>$99%. This second-stage FCN can now focus on more detailed segmentation of the organs. We respectively utilize training and validation sets consisting of 281 and 50 clinical CT images. Our hierarchical approach provides an improved Dice score of 7.5 percentage points per organ on average in our validation set. We furthermore test our models on a completely unseen data collection acquired at a different hospital that includes 150 CT scans with three anatomical labels (liver, spleen, and pancreas). In such challenging organs as the pancreas, our hierarchical approach improves the mean Dice score from 68.5 to 82.2%, achieving the highest reported average score on this dataset.

References in corpus (6)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
• Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation
• A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans

Cited by in corpus (15)

• Attention U-Net: Learning Where to Look for the Pancreas
• Abdominal multi-organ segmentation with organ-attention networks and statistical fusion
• UNETR: Transformers for 3D Medical Image Segmentation
• A two-stage 3D Unet framework for multi-class segmentation on full resolution image
• Deep Learning Based 3D Segmentation: A Survey
• Towards Automatic Abdominal Multi-Organ Segmentation in Dual Energy CT using Cascaded 3D Fully Convolutional Network
• Semi-Supervised Multi-Organ Segmentation via Deep Multi-Planar Co-Training
• Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation
• Interactive Deep Refinement Network for Medical Image Segmentation
• SegTHOR: Segmentation of Thoracic Organs at Risk in CT images
• 3D U${}^2$-Net: A 3D Universal U-Net for Multi-Domain Medical Image Segmentation
• Deep Distance Transform for Tubular Structure Segmentation in CT Scans
• Training Multi-organ Segmentation Networks with Sample Selection by Relaxed Upper Confident Bound
• Anatomically Consistent Segmentation of Organs at Risk in MRI with Convolutional Neural Networks
• Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review