U-Net and its variants for medical image segmentation: theory and applications

Abstract

U-net is an image segmentation technique developed primarily for medical image analysis that can precisely segment images using a scarce amount of training data. These traits provide U-net with a very high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in all major image modalities from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. As the potential of U-net is still increasing, in this review we look at the various developments that have been made in the U-net architecture and provide observations on recent trends. We examine the various innovations that have been made in deep learning and discuss how these tools facilitate U-net. Furthermore, we look at image modalities and application areas where U-net has been applied.

42 pages, in IEEE Access

References in corpus (38)

• A Survey on Deep Learning in Medical Image Analysis
• Conditional Generative Adversarial Nets
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Attention U-Net: Learning Where to Look for the Pancreas
• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Fully Convolutional Networks for Semantic Segmentation
• MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation
• CE-Net: Context Encoder Network for 2D Medical Image Segmentation
• Generative Adversarial Network in Medical Imaging: A Review
• Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation and Diagnosis for COVID-19
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• DUNet: A deformable network for retinal vessel segmentation
• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• Boundary loss for highly unbalanced segmentation
• Modified U-Net (mU-Net) with Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images
• Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
• Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation
• W-Net: A Deep Model for Fully Unsupervised Image Segmentation
• Residual Attention U-Net for Automated Multi-Class Segmentation of COVID-19 Chest CT Images
• PanNuke Dataset Extension, Insights and Baselines
• Deep Q Learning Driven CT Pancreas Segmentation with Geometry-Aware U-Net
• Multi-Scale Supervised 3D U-Net for Kidneys and Kidney Tumor Segmentation
• Attention U-Net Based Adversarial Architectures for Chest X-ray Lung Segmentation
• COVID-19 Chest CT Image Segmentation -- A Deep Convolutional Neural Network Solution
• H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes
• COVID_MTNet: COVID-19 Detection with Multi-Task Deep Learning Approaches
• T-Net: Nested encoder-decoder architecture for the main vessel segmentation in coronary angiography
• Improving the Segmentation of Anatomical Structures in Chest Radiographs using U-Net with an ImageNet Pre-trained Encoder
• U2-Net: A Bayesian U-Net model with epistemic uncertainty feedback for photoreceptor layer segmentation in pathological OCT scans
• Pulmonary Vessel Segmentation based on Orthogonal Fused U-Net++ of Chest CT Images
• Automatic quantification of the LV function and mass: a deep learning approach for cardiovascular MRI
• Random 2.5D U-net for Fully 3D Segmentation
• MSU-Net: Multiscale Statistical U-Net for Real-time 3D Cardiac MRI Video Segmentation
• Radiologist-level stroke classification on non-contrast CT scans with Deep U-Net
• FU-net: Multi-class Image Segmentation Using Feedback Weighted U-net
• Prostate Gland Segmentation in Histology Images via Residual and Multi-Resolution U-Net
• Deep Residual 3D U-Net for Joint Segmentation and Texture Classification of Nodules in Lung

Cited by in corpus (48)

• Comparing YOLOv8 and Mask R-CNN for instance segmentation in complex orchard environments
• Rethinking the Unpretentious U-net for Medical Ultrasound Image Segmentation
• Trends in deep learning for medical hyperspectral image analysis
• A Survey of Deep Learning Techniques for the Analysis of COVID-19 and their usability for Detecting Omicron
• Implementation of a Modified U-Net for Medical Image Segmentation on Edge Devices
• Application of belief functions to medical image segmentation: A review
• Advanced sleep spindle identification with neural networks
• PocketNet: A Smaller Neural Network for Medical Image Analysis
• Reviewing Intelligent Cinematography: AI research for camera-based video production
• Deep learning-based GTV contouring modeling inter- and intra- observer variability in sarcomas
• Label-free segmentation from cardiac ultrasound using self-supervised learning
• Transfer Learning U-Net Deep Learning for Lung Ultrasound Segmentation
• TMS-Net: A Segmentation Network Coupled With A Run-time Quality Control Method For Robust Cardiac Image Segmentation
• Res-U2Net: Untrained Deep Learning for Phase Retrieval and Image Reconstruction
• Trichomonas Vaginalis Segmentation in Microscope Images
• Autofocusing+: Noise-Resilient Motion Correction in Magnetic Resonance Imaging
• FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training
• A Densely Interconnected Network for Deep Learning Accelerated MRI
• Transfer learning-based physics-informed convolutional neural network for simulating flow in porous media with time-varying controls
• Evaluation of Data Augmentation and Loss Functions in Semantic Image Segmentation for Drilling Tool Wear Detection
• Amplitude SAR Imagery Splicing Localization
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance
• Muographic Image Upsampling with Machine Learning for Built Infrastructure Applications
• A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers
• MAGIC: Microlensing Analysis Guided by Intelligent Computation
• Dynamic hysteresis model of grain-oriented ferromagnetic material using neural operators
• U-Net-and-a-half: Convolutional network for biomedical image segmentation using multiple expert-driven annotations
• LGPS: A Lightweight GAN-Based Approach for Polyp Segmentation in Colonoscopy Images
• CurriculumLoc: Enhancing Cross-Domain Geolocalization through Multi-Stage Refinement
• Automatic detection of single-electron regime of quantum dots and definition of virtual gates using U-Net and clustering
• Classification of colorectal primer carcinoma from normal colon with mid-infrared spectra
• Morphomics via Next-generation Electron Microscopy
• Self-attention U-Net decoder for toric codes
• Image Prior and Posterior Conditional Probability Representation for Efficient Damage Assessment
• Mapping neutral islands during end stages of reionization with photometric intergalactic medium tomography
• DeepFDR: A Deep Learning-based False Discovery Rate Control Method for Neuroimaging Data
• COVID-19 Detection Using Segmentation, Region Extraction and Classification Pipeline
• Multi-Resolution Feature Fusion U-Net for Magnetic Resonance Imaging Segmentation
• Predicting large scale cosmological structure evolution with generative adversarial network-based autoencoders
• DDU-Net: A Domain Decomposition-Based CNN for High-Resolution Image Segmentation on Multiple GPUs
• Full-scale Representation Guided Network for Retinal Vessel Segmentation
• Automated Dental Caries Segmentation in Panoramic Radiographs Using Dual-Stage Deep Learning
• Transforming Hyperspectral Images Into Chemical Maps: A Novel End-to-End Deep Learning Approach
• PDFNet: Pointwise Dense Flow Network for Urban-Scene Segmentation
• BP-Net: Efficient Deep Learning for Continuous Arterial Blood Pressure Estimation using Photoplethysmogram
• Deep Learning for Reaction-Diffusion Glioma Growth Modelling: Towards a Fully Personalised Model?
• Stereo Video Reconstruction Without Explicit Depth Maps for Endoscopic Surgery