MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning

Abstract

Segmentation of organs or lesions from medical images plays an essential role in many clinical applications such as diagnosis and treatment planning. Though Convolutional Neural Networks (CNN) have achieved the state-of-the-art performance for automatic segmentation, they are often limited by the lack of clinically acceptable accuracy and robustness in complex cases. Therefore, interactive segmentation is a practical alternative to these methods. However, traditional interactive segmentation methods require a large amount of user interactions, and recently proposed CNN-based interactive segmentation methods are limited by poor performance on previously unseen objects. To solve these problems, we propose a novel deep learning-based interactive segmentation method that not only has high efficiency due to only requiring clicks as user inputs but also generalizes well to a range of previously unseen objects. Specifically, we first encode user-provided interior margin points via our proposed exponentialized geodesic distance that enables a CNN to achieve a good initial segmentation result of both previously seen and unseen objects, then we use a novel information fusion method that combines the initial segmentation with only few additional user clicks to efficiently obtain a refined segmentation. We validated our proposed framework through extensive experiments on 2D and 3D medical image segmentation tasks with a wide range of previous unseen objects that were not present in the training set. Experimental results showed that our proposed framework 1) achieves accurate results with fewer user interactions and less time compared with state-of-the-art interactive frameworks and 2) generalizes well to previously unseen objects.

17 pages, 16 figures

References in corpus (8)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Bottom-up Object Detection by Grouping Extreme and Center Points
• Interactive segmentation of medical images through fully convolutional neural networks
• Uncertainty-Guided Efficient Interactive Refinement of Fetal Brain Segmentation from Stacks of MRI Slices
• Weakly supervised segmentation from extreme points
• Deep GrabCut for Object Selection
• Going to Extremes: Weakly Supervised Medical Image Segmentation

Cited by in corpus (11)

• Segment Anything in Medical Images
• WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image
• MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
• SATS: Self-Attention Transfer for Continual Semantic Segmentation
• Multimodal Interactive Lung Lesion Segmentation: A Framework for Annotating PET/CT Images based on Physiological and Anatomical Cues
• UM-CAM: Uncertainty-weighted Multi-resolution Class Activation Maps for Weakly-supervised Fetal Brain Segmentation
• Minimally Interactive Segmentation of Soft-Tissue Tumors on CT and MRI using Deep Learning
• Guiding the Guidance: A Comparative Analysis of User Guidance Signals for Interactive Segmentation of Volumetric Images
• Adaptive Multi-scale Online Likelihood Network for AI-assisted Interactive Segmentation
• DEEPBEAS3D: Deep Learning and B-Spline Explicit Active Surfaces