Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function

Abstract

Deep neural networks have demonstrated very promising performance on accurate segmentation of challenging organs (e.g., pancreas) in abdominal CT and MRI scans. The current deep learning approaches conduct pancreas segmentation by processing sequences of 2D image slices independently through deep, dense per-pixel masking for each image, without explicitly enforcing spatial consistency constraint on segmentation of successive slices. We propose a new convolutional/recurrent neural network architecture to address the contextual learning and segmentation consistency problem. A deep convolutional sub-network is first designed and pre-trained from scratch. The output layer of this network module is then connected to recurrent layers and can be fine-tuned for contextual learning, in an end-to-end manner. Our recurrent sub-network is a type of Long short-term memory (LSTM) network that performs segmentation on an image by integrating its neighboring slice segmentation predictions, in the form of a dependent sequence processing. Additionally, a novel segmentation-direct loss function (named Jaccard Loss) is proposed and deep networks are trained to optimize Jaccard Index (JI) directly. Extensive experiments are conducted to validate our proposed deep models, on quantitative pancreas segmentation using both CT and MRI scans. Our method outperforms the state-of-the-art work on CT [11] and MRI pancreas segmentation [1], respectively.

arXiv comment8 pages, 7 figures, accepted to Medical Image Computing and Computer Assisted Interventions Conference (MICCAI) 2017

References in corpus (4)

• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Holistically-Nested Edge Detection
• DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
• Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation

Cited by in corpus (27)

• Attention U-Net: Learning Where to Look for the Pancreas
• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
• Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information
• Fully Automated Multi-Organ Segmentation in Abdominal Magnetic Resonance Imaging with Deep Neural Networks
• Pancreas segmentation with probabilistic map guided bi-directional recurrent UNet
• Deep learning in radiology: an overview of the concepts and a survey of the state of the art
• Pancreas Segmentation in CT and MRI Images via Domain Specific Network Designing and Recurrent Neural Contextual Learning
• Hierarchical 3D Feature Learning for Pancreas Segmentation
• Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST
• V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Interactive Medical Image Segmentation via Point-Based Interaction and Sequential Patch Learning
• Computer-aided Detection of Squamous Carcinoma of the Cervix in Whole Slide Images
• Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data
• Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation
• Segmentation for Classification of Screening Pancreatic Neuroendocrine Tumors
• A Model-Driven Stack-Based Fully Convolutional Network for Pancreas Segmentation
• A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation
• Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks
• Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST
• Small Organ Segmentation in Whole-body MRI using a Two-stage FCN and Weighting Schemes
• Recurrent U-net for automatic pelvic floor muscle segmentation on 3D ultrasound
• Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation
• Bidirectional RNN-based Few Shot Learning for 3D Medical Image Segmentation
• Selective Information Passing for MR/CT Image Segmentation
• Splenomegaly Segmentation on Multi-modal MRI using Deep Convolutional Networks
• Tissue characterization based on the analysis on i3DUS data for diagnosis support in neurosurgery