Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function

Abstract

Deep neural networks have demonstrated very promising performance on accurate segmentation of challenging organs (e.g., pancreas) in abdominal CT and MRI scans. The current deep learning approaches conduct pancreas segmentation by processing sequences of 2D image slices independently through deep, dense per-pixel masking for each image, without explicitly enforcing spatial consistency constraint on segmentation of successive slices. We propose a new convolutional/recurrent neural network architecture to address the contextual learning and segmentation consistency problem. A deep convolutional sub-network is first designed and pre-trained from scratch. The output layer of this network module is then connected to recurrent layers and can be fine-tuned for contextual learning, in an end-to-end manner. Our recurrent sub-network is a type of Long short-term memory (LSTM) network that performs segmentation on an image by integrating its neighboring slice segmentation predictions, in the form of a dependent sequence processing. Additionally, a novel segmentation-direct loss function (named Jaccard Loss) is proposed and deep networks are trained to optimize Jaccard Index (JI) directly. Extensive experiments are conducted to validate our proposed deep models, on quantitative pancreas segmentation using both CT and MRI scans. Our method outperforms the state-of-the-art work on CT [11] and MRI pancreas segmentation [1], respectively.

8 pages, 7 figures, accepted to Medical Image Computing and Computer Assisted Interventions Conference (MICCAI) 2017

References in corpus (2)

• DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
• Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation

Cited by in corpus (10)

• Hierarchical 3D Feature Learning for Pancreas Segmentation
• Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Computer-aided Detection of Squamous Carcinoma of the Cervix in Whole Slide Images
• Segmentation for Classification of Screening Pancreatic Neuroendocrine Tumors
• Recurrent U-net for automatic pelvic floor muscle segmentation on 3D ultrasound
• Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation
• Bidirectional RNN-based Few Shot Learning for 3D Medical Image Segmentation
• Tissue characterization based on the analysis on i3DUS data for diagnosis support in neurosurgery
• Selective Information Passing for MR/CT Image Segmentation