MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation

Abstract

In recent years Deep Learning has brought about a breakthrough in Medical Image Segmentation. U-Net is the most prominent deep network in this regard, which has been the most popular architecture in the medical imaging community. Despite outstanding overall performance in segmenting multimodal medical images, from extensive experimentations on challenging datasets, we found out that the classical U-Net architecture seems to be lacking in certain aspects. Therefore, we propose some modifications to improve upon the already state-of-the-art U-Net model. Hence, following the modifications we develop a novel architecture MultiResUNet as the potential successor to the successful U-Net architecture. We have compared our proposed architecture MultiResUNet with the classical U-Net on a vast repertoire of multimodal medical images. Albeit slight improvements in the cases of ideal images, a remarkable gain in performance has been attained for challenging images. We have evaluated our model on five different datasets, each with their own unique challenges, and have obtained a relative improvement in performance of 10.15%, 5.07%, 2.63%, 1.41%, and 0.62% respectively.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Scikit-learn: Machine Learning in Python
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• A Survey on Deep Learning in Medical Image Analysis
• TensorFlow: A system for large-scale machine learning
• An overview of gradient descent optimization algorithms
• Fully Convolutional Networks for Semantic Segmentation
• Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
• Medical Image Analysis using Convolutional Neural Networks: A Review
• Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections
• The Importance of Skip Connections in Biomedical Image Segmentation

Cited by in corpus (80)

• U-Net and its variants for medical image segmentation: theory and applications
• Medical Image Segmentation Using Deep Learning: A Survey
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images
• A2-FPN for Semantic Segmentation of Fine-Resolution Remotely Sensed Images
• DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentation
• Boundary-aware Context Neural Network for Medical Image Segmentation
• Modality specific U-Net variants for biomedical image segmentation: A survey
• Dual Cross-Attention for Medical Image Segmentation
• Residual Attention U-Net for Automated Multi-Class Segmentation of COVID-19 Chest CT Images
• Fully-automated Body Composition Analysis in Routine CT Imaging Using 3D Semantic Segmentation Convolutional Neural Networks
• MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning
• A Data-Driven Approach to Full-Field Damage and Failure Pattern Prediction in Microstructure-Dependent Composites using Deep Learning
• R2U++: A Multiscale Recurrent Residual U-Net with Dense Skip Connections for Medical Image Segmentation
• Review: Deep Learning in Electron Microscopy
• MFSNet: A Multi Focus Segmentation Network for Skin Lesion Segmentation
• A Shallow U-Net Architecture for Reliably Predicting Blood Pressure (BP) from Photoplethysmogram (PPG) and Electrocardiogram (ECG) Signals
• Effects of Image Size on Deep Learning
• EDITH :ECG biometrics aided by Deep learning for reliable Individual auTHentication
• CovTANet: A Hybrid Tri-level Attention Based Network for Lesion Segmentation, Diagnosis, and Severity Prediction of COVID-19 Chest CT Scans
• Inter- and intra-uncertainty based feature aggregation model for semi-supervised histopathology image segmentation
• Application of belief functions to medical image segmentation: A review
• A global method to identify trees outside of closed-canopy forests with medium-resolution satellite imagery
• DoubleU-NetPlus: A Novel Attention and Context Guided Dual U-Net with Multi-Scale Residual Feature Fusion Network for Semantic Segmentation of Medical Images
• Meta Ordinal Regression Forest for Medical Image Classification with Ordinal Labels
• ipA-MedGAN: Inpainting of Arbitrary Regions in Medical Imaging
• Droplet size distribution in a swirl airstream using in-line holography technique
• SkipConvNet: Skip Convolutional Neural Network for Speech Dereverberation using Optimally Smoothed Spectral Mapping
• SkipConvGAN: Monaural Speech Dereverberation using Generative Adversarial Networks via Complex Time-Frequency Masking
• ARPM-net: A novel CNN-based adversarial method with Markov Random Field enhancement for prostate and organs at risk segmentation in pelvic CT images
• A Multi-scale CNN-CRF Framework for Environmental Microorganism Image Segmentation
• Edge-aware Feature Aggregation Network for Polyp Segmentation
• MLN-net: A multi-source medical image segmentation method for clustered microcalcifications using multiple layer normalization
• Physical Accuracy of Deep Neural Networks for 2D and 3D Multi-Mineral Segmentation of Rock micro-CT Images
• Hybrid Skip: A Biologically Inspired Skip Connection for the UNet Architecture
• Category Guided Attention Network for Brain Tumor Segmentation in MRI
• DONet: Dual Objective Networks for Skin Lesion Segmentation
• Facial UV Map Completion for Pose-invariant Face Recognition: A Novel Adversarial Approach based on Coupled Attention Residual UNets
• CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentation
• PPG2ABP: Translating Photoplethysmogram (PPG) Signals to Arterial Blood Pressure (ABP) Waveforms using Fully Convolutional Neural Networks
• Prostate Gland Segmentation in Histology Images via Residual and Multi-Resolution U-Net
• Deep learning in magnetic resonance prostate segmentation: A review and a new perspective
• A Region of Interest Focused Triple UNet Architecture for Skin Lesion Segmentation
• A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation
• Efficient Perception, Planning, and Control Algorithm for Vision-Based Automated Vehicles
• Deep Bayesian segmentation for colon polyps: Well-calibrated predictions in medical imaging
• Segmentation method of U-net sheet metal engineering drawing based on CBAM attention mechanism
• Automatic acute ischemic stroke lesion segmentation using semi-supervised learning
• Preserving Dense Features for Ki67 Nuclei Detection
• Heuristical Comparison of Vision Transformers Against Convolutional Neural Networks for Semantic Segmentation on Remote Sensing Imagery
• AG-CUResNeSt: A Novel Method for Colon Polyp Segmentation
• MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures
• An Efficient Dual-Line Decoder Network with Multi-Scale Convolutional Attention for Multi-organ Segmentation
• Deeply Cascaded U-Net for Multi-Task Image Processing
• Segmentation of Surgical Instruments for Minimally-Invasive Robot-Assisted Procedures Using Generative Deep Neural Networks
• Learning the Universe: Learning to Optimize Cosmic Initial Conditions with Non-Differentiable Structure Formation Models
• HybridNetSeg: A Compact Hybrid Network for Retinal Vessel Segmentation
• U-Net-and-a-half: Convolutional network for biomedical image segmentation using multiple expert-driven annotations
• U-net Based Direct-path Dominance Test for Robust Direction-of-arrival Estimation
• Very High Resolution Land Cover Mapping of Urban Areas at Global Scale with Convolutional Neural Networks
• MS-NAS: Multi-Scale Neural Architecture Search for Medical Image Segmentation
• Slim U-Net: Efficient Anatomical Feature Preserving U-net Architecture for Ultrasound Image Segmentation
• CovSegNet: A Multi Encoder-Decoder Architecture for Improved Lesion Segmentation of COVID-19 Chest CT Scans
• SkiNet: A Deep Learning Solution for Skin Lesion Diagnosis with Uncertainty Estimation and Explainability
• Fully Automated and Standardized Segmentation of Adipose Tissue Compartments by Deep Learning in Three-dimensional Whole-body MRI of Epidemiological Cohort Studies
• FDNet: Feature Decoupled Segmentation Network for Tooth CBCT Image
• Janssen 2.0: Audio Inpainting in the Time-frequency Domain
• Revealing the Evolution of Order in Materials Microstructures Using Multi-Modal Computer Vision
• Cross-Modal Self-Attention Distillation for Prostate Cancer Segmentation
• UXNet: Searching Multi-level Feature Aggregation for 3D Medical Image Segmentation
• J-Net: Randomly weighted U-Net for audio source separation
• Video Compression Coding via Colorization: A Generative Adversarial Network (GAN)-Based Approach
• Segmentation of Lung Tumor from CT Images using Deep Supervision
• Segmentation and Generation of Magnetic Resonance Images by Deep Neural Networks
• Content-Aware Convolutional Neural Networks
• Knowledge distillation from multi-modal to mono-modal segmentation networks
• Multi-Resolution Feature Fusion U-Net for Magnetic Resonance Imaging Segmentation
• RectiNet-v2: A stacked network architecture for document image dewarping
• Deep Class-Specific Affinity-Guided Convolutional Network for Multimodal Unpaired Image Segmentation
• DeepPyram: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos