Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation

Abstract

Segmentation of 3D images is a fundamental problem in biomedical image analysis. Deep learning (DL) approaches have achieved state-of-the-art segmentation perfor- mance. To exploit the 3D contexts using neural networks, known DL segmentation methods, including 3D convolution, 2D convolution on planes orthogonal to 2D image slices, and LSTM in multiple directions, all suffer incompatibility with the highly anisotropic dimensions in common 3D biomedical images. In this paper, we propose a new DL framework for 3D image segmentation, based on a com- bination of a fully convolutional network (FCN) and a recurrent neural network (RNN), which are responsible for exploiting the intra-slice and inter-slice contexts, respectively. To our best knowledge, this is the first DL framework for 3D image segmentation that explicitly leverages 3D image anisotropism. Evaluating using a dataset from the ISBI Neuronal Structure Segmentation Challenge and in-house image stacks for 3D fungus segmentation, our approach achieves promising results comparing to the known DL-based 3D segmentation approaches.

References in corpus (3)

• DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation
• Deep Learning for Medical Image Segmentation
• rnn : Recurrent Library for Torch

Cited by in corpus (28)

• A Survey on Deep Learning in Medical Image Analysis
• Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network
• FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics
• Fully Convolutional Network Ensembles for White Matter Hyperintensities Segmentation in MR Images
• Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information
• A Deep Learning System That Generates Quantitative CT Reports for Diagnosing Pulmonary Tuberculosis
• 3D RoI-aware U-Net for Accurate and Efficient Colorectal Tumor Segmentation
• Deep learning in biomedical optics
• A Two-step-training Deep Learning Framework for Real-time Computational Imaging without Physics Priors
• Recurrent neural network-based volumetric fluorescence microscopy
• Pancreas Segmentation in CT and MRI Images via Domain Specific Network Designing and Recurrent Neural Contextual Learning
• Unsupervised inter-frame motion correction for whole-body dynamic PET using convolutional long short-term memory in a convolutional neural network
• Joint Sequence Learning and Cross-Modality Convolution for 3D Biomedical Segmentation
• Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning
• Morphological Error Detection in 3D Segmentations
• PSIque: Next Sequence Prediction of Satellite Images using a Convolutional Sequence-to-Sequence Network
• Automatic Lymphocyte Detection in H&E Images with Deep Neural Networks
• A New Ensemble Learning Framework for 3D Biomedical Image Segmentation
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• 4D Deep Learning for Multiple Sclerosis Lesion Activity Segmentation
• A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans
• Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks
• V-FCNN: Volumetric Fully Convolution Neural Network For Automatic Atrial Segmentation
• Recurrent U-net for automatic pelvic floor muscle segmentation on 3D ultrasound
• A Generative Adversarial Model for Right Ventricle Segmentation
• Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
• Semantic Segmentation of highly class imbalanced fully labelled 3D volumetric biomedical images and unsupervised Domain Adaptation of the pre-trained Segmentation Network to segment another fully unlabelled Biomedical 3D Image stack
• Multi-stage Multi-recursive-input Fully Convolutional Networks for Neuronal Boundary Detection