Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation

Abstract

Accurate and automatic organ segmentation from 3D radiological scans is an important yet challenging problem for medical image analysis. Specifically, the pancreas demonstrates very high inter-patient anatomical variability in both its shape and volume. In this paper, we present an automated system using 3D computed tomography (CT) volumes via a two-stage cascaded approach: pancreas localization and segmentation. For the first step, we localize the pancreas from the entire 3D CT scan, providing a reliable bounding box for the more refined segmentation step. We introduce a fully deep-learning approach, based on an efficient application of holistically-nested convolutional networks (HNNs) on the three orthogonal axial, sagittal, and coronal views. The resulting HNN per-pixel probability maps are then fused using pooling to reliably produce a 3D bounding box of the pancreas that maximizes the recall. We show that our introduced localizer compares favorably to both a conventional non-deep-learning method and a recent hybrid approach based on spatial aggregation of superpixels using random forest classification. The second, segmentation, phase operates within the computed bounding box and integrates semantic mid-level cues of deeply-learned organ interior and boundary maps, obtained by two additional and separate realizations of HNNs. By integrating these two mid-level cues, our method is capable of generating boundary-preserving pixel-wise class label maps that result in the final pancreas segmentation. Quantitative evaluation is performed on a publicly available dataset of 82 patient CT scans using 4-fold cross-validation (CV). We achieve a Dice similarity coefficient (DSC) of 81.27+/-6.27% in validation, which significantly outperforms previous state-of-the art methods that report DSCs of 71.80+/-10.70% and 78.01+/-8.20%, respectively, using the same dataset.

This version was submitted to IEEE Trans. on Medical Imaging on Dec. 18th, 2016. The content of this article is covered by US Patent Applications of 62/345,606# and 62/450,681#

References in corpus (9)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Deeply-Supervised Nets
• Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
• Improving Computer-aided Detection using Convolutional Neural Networks and Random View Aggregation
• Holistically-Nested Edge Detection
• Multi-view Convolutional Neural Networks for 3D Shape Recognition
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
• 3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes
• Dense Volume-to-Volume Vascular Boundary Detection

Cited by in corpus (4)

• An application of cascaded 3D fully convolutional networks for medical image segmentation
• Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function
• Hierarchical 3D fully convolutional networks for multi-organ segmentation
• Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation