DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation

Abstract

Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications. Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods. However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use. We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy. We use one CNN to obtain an initial automatic segmentation, on which user interactions are added to indicate mis-segmentations. Another CNN takes as input the user interactions with the initial segmentation and gives a refined result. We propose to combine user interactions with CNNs through geodesic distance transforms, and propose a resolution-preserving network that gives a better dense prediction. In addition, we integrate user interactions as hard constraints into a back-propagatable Conditional Random Field. We validated the proposed framework in the context of 2D placenta segmentation from fetal MRI and 3D brain tumor segmentation from FLAIR images. Experimental results show our method achieves a large improvement from automatic CNNs, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.

arXiv comment14 pages, 15 figures

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Survey on Deep Learning in Medical Image Analysis
• TensorFlow: A system for large-scale machine learning
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Brain Tumor Segmentation with Deep Neural Networks
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Fully Convolutional Networks for Semantic Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• Learning Graphical Model Parameters with Approximate Marginal Inference
• Scalable multimodal convolutional networks for brain tumour segmentation
• End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks

Cited by in corpus (56)

• Segment Anything in Medical Images
• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• Medical Image Segmentation Using Deep Learning: A Survey
• CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation
• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis
• NiftyNet: a deep-learning platform for medical imaging
• Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• Boundary-aware Context Neural Network for Medical Image Segmentation
• WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image
• MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning
• MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images
• CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions
• Manual segmentation versus semi-automated segmentation for quantifying vestibular schwannoma volume on MRI
• Interactive segmentation of medical images through fully convolutional neural networks
• A Physics-Guided Modular Deep-Learning Based Automated Framework for Tumor Segmentation in PET Images
• From IC Layout to Die Photo: A CNN-Based Data-Driven Approach
• Annotation-Efficient Learning for Medical Image Segmentation based on Noisy Pseudo Labels and Adversarial Learning
• Uncertainty-Guided Efficient Interactive Refinement of Fetal Brain Segmentation from Stacks of MRI Slices
• Curvature-based Feature Selection with Application in Classifying Electronic Health Records
• OneLabeler: A Flexible System for Building Data Labeling Tools
• Boundary-aware Supervoxel-level Iteratively Refined Interactive 3D Image Segmentation with Multi-agent Reinforcement Learning
• DeepEdit: Deep Editable Learning for Interactive Segmentation of 3D Medical Images
• Scribble-based Domain Adaptation via Co-segmentation
• Hyper-Fusion Network for Semi-Automatic Segmentation of Skin Lesions
• Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation
• Automatic segmentation method of pelvic floor levator hiatus in ultrasound using a self-normalising neural network
• Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey
• FastGeodis: Fast Generalised Geodesic Distance Transform
• Weakly supervised segmentation from extreme points
• Geodesic Paths for Image Segmentation with Implicit Region-based Homogeneity Enhancement
• SIGAN: A Novel Image Generation Method for Solar Cell Defect Segmentation and Augmentation
• Multimodal Interactive Lung Lesion Segmentation: A Framework for Annotating PET/CT Images based on Physiological and Anatomical Cues
• DINs: Deep Interactive Networks for Neurofibroma Segmentation in Neurofibromatosis Type 1 on Whole-Body MRI
• Adaptive Multi-scale Online Likelihood Network for AI-assisted Interactive Segmentation
• PlaNet-S: Automatic Semantic Segmentation of Placenta
• A hybrid multi-object segmentation framework with model-based B-splines for microbial single cell analysis
• Interactive Medical Image Segmentation with Self-Adaptive Confidence Calibration
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding
• OOD-SEG: Exploiting out-of-distribution detection techniques for learning image segmentation from sparse multi-class positive-only annotations
• Iterative Interaction Training for Segmentation Editing Networks
• DEEPBEAS3D: Deep Learning and B-Spline Explicit Active Surfaces
• HAISTA-NET: Human Assisted Instance Segmentation Through Attention
• ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentation
• SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Dual Shape Guided Segmentation Network for Organs-at-Risk in Head and Neck CT Images
• Vessel-CAPTCHA: an efficient learning framework for vessel annotation and segmentation
• Interactive Radiotherapy Target Delineation with 3D-Fused Context Propagation
• Iteratively-Refined Interactive 3D Medical Image Segmentation with Multi-Agent Reinforcement Learning
• Is Complete Labeling Necessary? Understanding Active Learning in Longitudinal Medical Imaging
• One-shot Weakly-Supervised Segmentation in Medical Images
• A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation
• End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention
• Label tree semantic losses for rich multi-class medical image segmentation
• Localized Interactive Instance Segmentation