3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation

Abstract

This paper introduces a network for volumetric segmentation that learns from sparsely annotated volumetric images. We outline two attractive use cases of this method: (1) In a semi-automated setup, the user annotates some slices in the volume to be segmented. The network learns from these sparse annotations and provides a dense 3D segmentation. (2) In a fully-automated setup, we assume that a representative, sparsely annotated training set exists. Trained on this data set, the network densely segments new volumetric images. The proposed network extends the previous u-net architecture from Ronneberger et al. by replacing all 2D operations with their 3D counterparts. The implementation performs on-the-fly elastic deformations for efficient data augmentation during training. It is trained end-to-end from scratch, i.e., no pre-trained network is required. We test the performance of the proposed method on a complex, highly variable 3D structure, the Xenopus kidney, and achieve good results for both use cases.

Conditionally accepted for MICCAI 2016

References in corpus (3)

• Rethinking the Inception Architecture for Computer Vision
• Hough-CNN: Deep Learning for Segmentation of Deep Brain Regions in MRI and Ultrasound
• Deep End2End Voxel2Voxel Prediction

Cited by in corpus (125)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation
• Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
• TractSeg - Fast and accurate white matter tract segmentation
• RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans
• VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm
• An application of cascaded 3D fully convolutional networks for medical image segmentation
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics
• Iterative fully convolutional neural networks for automatic vertebra segmentation and identification
• Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks
• MISSFormer: An Effective Medical Image Segmentation Transformer
• End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reduction
• CNN-based Segmentation of Medical Imaging Data
• MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning
• Multi-Scale Supervised 3D U-Net for Kidneys and Kidney Tumor Segmentation
• Hierarchical 3D fully convolutional networks for multi-organ segmentation
• Adapting Mask-RCNN for Automatic Nucleus Segmentation
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
• FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
• T-Net: Nested encoder-decoder architecture for the main vessel segmentation in coronary angiography
• Effects of Image Size on Deep Learning
• Deep Learning for Lung Cancer Detection: Tackling the Kaggle Data Science Bowl 2017 Challenge
• Cloud and Cloud Shadow Segmentation for Remote Sensing Imagery via Filtered Jaccard Loss Function and Parametric Augmentation
• Interactive segmentation of medical images through fully convolutional neural networks
• A two-stage 3D Unet framework for multi-class segmentation on full resolution image
• Gastric histopathology image segmentation using a hierarchical conditional random field
• Algorithmically-Consistent Deep Learning Frameworks for Structural Topology Optimization
• Quantitative analysis of patch-based fully convolutional neural networks for tissue segmentation on brain magnetic resonance imaging
• Direct observation of three-dimensional atomic structure of twinned metallic nanoparticles and their catalytic properties
• Tangent Convolutions for Dense Prediction in 3D
• Task adapted reconstruction for inverse problems
• SpecTr: Spectral Transformer for Hyperspectral Pathology Image Segmentation
• Online Normalization for Training Neural Networks
• A Deep Learning Approach for Pose Estimation from Volumetric OCT Data
• Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction
• Stacked U-Nets: A No-Frills Approach to Natural Image Segmentation
• On the influence of Dice loss function in multi-class organ segmentation of abdominal CT using 3D fully convolutional networks
• Pancreas Segmentation in CT and MRI Images via Domain Specific Network Designing and Recurrent Neural Contextual Learning
• TFLMS: Large Model Support in TensorFlow by Graph Rewriting
• Precise Estimation of Renal Vascular Dominant Regions Using Spatially Aware Fully Convolutional Networks, Tensor-Cut and Voronoi Diagrams
• Multi Modal Convolutional Neural Networks for Brain Tumor Segmentation
• Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
• Automatic Volumetric Segmentation of Additive Manufacturing Defects with 3D U-Net
• Fully-Convolutional Point Networks for Large-Scale Point Clouds
• Cardiac Segmentation on CT Images through Shape-Aware Contour Attentions
• Domain specific cues improve robustness of deep learning based segmentation of ct volumes
• Deep Negative Volume Segmentation
• Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks
• Beyond the Pixel-Wise Loss for Topology-Aware Delineation
• Global Planar Convolutions for improved context aggregation in Brain Tumor Segmentation
• Modality-Pairing Learning for Brain Tumor Segmentation
• Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach
• Extracting 3D Vascular Structures from Microscopy Images using Convolutional Recurrent Networks
• A Survey on Biomedical Image Captioning
• Deep Segmentation and Registration in X-Ray Angiography Video
• Deep cross-modality (MR-CT) educed distillation learning for cone beam CT lung tumor segmentation
• A New Ensemble Learning Framework for 3D Biomedical Image Segmentation
• Interactive Medical Image Segmentation via Point-Based Interaction and Sequential Patch Learning
• Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection
• Automated Olfactory Bulb Segmentation on High Resolutional T2-Weighted MRI
• A Transfer Learning Approach for Automated Segmentation of Prostate Whole Gland and Transition Zone in Diffusion Weighted MRI
• Self-Ensembling Contrastive Learning for Semi-Supervised Medical Image Segmentation
• Fidelity Imposed Network Edit (FINE) for Solving Ill-Posed Image Reconstruction
• Almost instant brain atlas segmentation for large-scale studies
• DeepAAA: clinically applicable and generalizable detection of abdominal aortic aneurysm using deep learning
• Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentation
• Spatially Localized Atlas Network Tiles Enables 3D Whole Brain Segmentation from Limited Data
• Point2Node: Correlation Learning of Dynamic-Node for Point Cloud Feature Modeling
• Collaborative Boundary-aware Context Encoding Networks for Error Map Prediction
• A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation
• Detecting Scatteredly-Distributed, Small, andCritically Important Objects in 3D OncologyImaging via Decision Stratification
• Organ at Risk Segmentation in Head and Neck CT Images by Using a Two-Stage Segmentation Framework Based on 3D U-Net
• Joint Cell Nuclei Detection and Segmentation in Microscopy Images Using 3D Convolutional Networks
• InverseNet: Solving Inverse Problems with Splitting Networks
• Critical Assessment of Transfer Learning for Medical Image Segmentation with Fully Convolutional Neural Networks
• An Overview and Case Study of the Clinical AI Model Development Life Cycle for Healthcare Systems
• Training Multi-organ Segmentation Networks with Sample Selection by Relaxed Upper Confident Bound
• Exploiting Shared Knowledge from Non-COVID Lesions for Annotation-Efficient COVID-19 CT Lung Infection Segmentation
• 3DQ: Compact Quantized Neural Networks for Volumetric Whole Brain Segmentation
• Invertible Residual Network with Regularization for Effective Medical Image Segmentation
• 3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks
• Dental pathology detection in 3D cone-beam CT
• Brain Tumor Segmentation using an Ensemble of 3D U-Nets and Overall Survival Prediction using Radiomic Features
• A Self-Adaptive Network For Multiple Sclerosis Lesion Segmentation From Multi-Contrast MRI With Various Imaging Protocols
• Lymph Node Gross Tumor Volume Detection and Segmentation via Distance-based Gating using 3D CT/PET Imaging in Radiotherapy
• Class balanced underwater object detection dataset generated by class-wise style augmentation
• Automatic Assessment of Full Left Ventricular Coverage in Cardiac Cine Magnetic Resonance Imaging with Fisher-Discriminative 3D CNN
• Anatomically Consistent Segmentation of Organs at Risk in MRI with Convolutional Neural Networks
• Automated Brain Tumour Segmentation Using Deep Fully Residual Convolutional Neural Networks
• Robustifying deep networks for image segmentation
• Multi-View Semantic Labeling of 3D Point Clouds for Automated Plant Phenotyping
• Prostate Segmentation using 2D Bridged U-net
• TOMAAT: volumetric medical image analysis as a cloud service
• Hyperspectral Image Classification with Spatial Consistence Using Fully Convolutional Spatial Propagation Network
• Dense Regression Activation Maps For Lesion Segmentation in CT scans of COVID-19 patients
• NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale
• Unsupervised COVID-19 Lesion Segmentation in CT Using Cycle Consistent Generative Adversarial Network
• Mass Segmentation in Automated 3-D Breast Ultrasound Using Dual-Path U-net
• HIVE-Net: Centerline-Aware HIerarchical View-Ensemble Convolutional Network for Mitochondria Segmentation in EM Images
• A reusable pipeline for large-scale fiber segmentation on unidirectional fiber beds using fully convolutional neural networks
• Tracing in 2D to Reduce the Annotation Effort for 3D Deep Delineation
• Nonlinear Collaborative Scheme for Deep Neural Networks
• Ischemic Stroke Lesion Segmentation in CT Perfusion Scans using Pyramid Pooling and Focal Loss
• Dense Multi-path U-Net for Ischemic Stroke Lesion Segmentation in Multiple Image Modalities
• A general approach to bridge the reality-gap
• Knee Cartilage Segmentation Using Diffusion-Weighted MRI
• Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion
• Enhancement Mask for Hippocampus Detection and Segmentation
• Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
• Tumor Delineation For Brain Radiosurgery by a ConvNet and Non-Uniform Patch Generation
• Automated Segmentation of Brain Gray Matter Nuclei on Quantitative Susceptibility Mapping Using Deep Convolutional Neural Network
• Holistic Decomposition Convolution for Effective Semantic Segmentation of 3D MR Images
• RootPainter3D: Interactive-machine-learning enables rapid and accurate contouring for radiotherapy
• A Study on Trees's Knots Prediction from their Bark Outer-Shape
• Automatically Segmenting the Left Atrium from Cardiac Images Using Successive 3D U-Nets and a Contour Loss
• A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching
• Voxel-level Siamese Representation Learning for Abdominal Multi-Organ Segmentation
• Advancement of Deep Learning in Pneumonia and Covid-19 Classification and Localization: A Qualitative and Quantitative Analysis
• 3D-OOCS: Learning Prostate Segmentation with Inductive Bias
• A Survey On 3D Inner Structure Prediction from its Outer Shape
• Semantic Segmentation of highly class imbalanced fully labelled 3D volumetric biomedical images and unsupervised Domain Adaptation of the pre-trained Segmentation Network to segment another fully unlabelled Biomedical 3D Image stack
• Rethinking Radiology: An Analysis of Different Approaches to BraTS