Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction

Abstract

Shortage of fully annotated datasets has been a limiting factor in developing deep learning based image segmentation algorithms and the problem becomes more pronounced in multi-organ segmentation. In this paper, we propose a unified training strategy that enables a novel multi-scale deep neural network to be trained on multiple partially labeled datasets for multi-organ segmentation. In addition, a new network architecture for multi-scale feature abstraction is proposed to integrate pyramid input and feature analysis into a U-shape pyramid structure. To bridge the semantic gap caused by directly merging features from different scales, an equal convolutional depth mechanism is introduced. Furthermore, we employ a deep supervision mechanism to refine the outputs in different scales. To fully leverage the segmentation features from all the scales, we design an adaptive weighting layer to fuse the outputs in an automatic fashion. All these mechanisms together are integrated into a Pyramid Input Pyramid Output Feature Abstraction Network (PIPO-FAN). Our proposed method was evaluated on four publicly available datasets, including BTCV, LiTS, KiTS and Spleen, where very promising performance has been achieved. The source code of this work is publicly shared at https://github.com/DIAL-RPI/PIPO-FAN for others to easily reproduce the work and build their own models with the introduced mechanisms.

Accepted for publication at IEEE Transactions on Medical Imaging

References in corpus (13)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Attention U-Net: Learning Where to Look for the Pancreas
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• The Liver Tumor Segmentation Benchmark (LiTS)
• An application of cascaded 3D fully convolutional networks for medical image segmentation
• The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes
• UNet++: A Nested U-Net Architecture for Medical Image Segmentation
• Abdominal multi-organ segmentation with organ-attention networks and statistical fusion
• Capsules for Object Segmentation
• A Method of Rapid Quantification of Patient-Specific Organ Dose for CT Using Coupled Deep-Learning based Multi-Organ Segmentation and GPU-accelerated Monte Carlo Dose Computing
• Hierarchical Convolutional-Deconvolutional Neural Networks for Automatic Liver and Tumor Segmentation
• Towards Automatic Abdominal Multi-Organ Segmentation in Dual Energy CT using Cascaded 3D Fully Convolutional Network
• Unified Multi-scale Feature Abstraction for Medical Image Segmentation