3D RoI-aware U-Net for Accurate and Efficient Colorectal Tumor Segmentation

Abstract

Segmentation of colorectal cancerous regions from 3D Magnetic Resonance (MR) images is a crucial procedure for radiotherapy which conventionally requires accurate delineation of tumour boundaries at an expense of labor, time and reproducibility. While deep learning based methods serve good baselines in 3D image segmentation tasks, small applicable patch size limits effective receptive field and degrades segmentation performance. In addition, Regions of interest (RoIs) localization from large whole volume 3D images serves as a preceding operation that brings about multiple benefits in terms of speed, target completeness, reduction of false positives. Distinct from sliding window or non-joint localization-segmentation based models, we propose a novel multitask framework referred to as 3D RoI-aware U-Net (3D RU-Net), for RoI localization and in-region segmentation where the two tasks share one backbone encoder network. With the region proposals from the encoder, we crop multi-level RoI in-region features from the encoder to form a GPU memory-efficient decoder for detailpreserving segmentation and therefore enlarged applicable volume size and effective receptive field. To effectively train the model, we designed a Dice formulated loss function for the global-to-local multi-task learning procedure. Based on the efficiency gains, we went on to ensemble models with different receptive fields to achieve even higher performance costing minor extra computational expensiveness. Extensive experiments were conducted on 64 cancerous cases with a four-fold cross-validation, and the results showed significant superiority in terms of accuracy and efficiency over conventional frameworks. In conclusion, the proposed method has a huge potential for extension to other 3D object segmentation tasks from medical images due to its inherent generalizability. The code for the proposed method is publicly available.

IEEE transactions on cybernetics (2020)

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Mixed Precision Training
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• Path Aggregation Network for Instance Segmentation
• A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations
• Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation
• Fully Automated Organ Segmentation in Male Pelvic CT Images
• Crossbar-Net: A Novel Convolutional Network for Kidney Tumor Segmentation in CT Images
• H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes
• Estimation of the volume of the left ventricle from MRI images using deep neural networks
• Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks

Cited by in corpus (8)

• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Deep learning-based GTV contouring modeling inter- and intra- observer variability in sarcomas
• Deep Negative Volume Segmentation
• Towards fully automated deep-learning-based brain tumor segmentation: is brain extraction still necessary?
• Weakly-supervised Learning For Catheter Segmentation in 3D Frustum Ultrasound
• MMFNet: A Multi-modality MRI Fusion Network for Segmentation of Nasopharyngeal Carcinoma
• Towards Label-Free 3D Segmentation of Optical Coherence Tomography Images of the Optic Nerve Head Using Deep Learning
• Crossover-Net: Leveraging the Vertical-Horizontal Crossover Relation for Robust Segmentation