TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation

Abstract

Pixel-wise image segmentation is demanding task in computer vision. Classical U-Net architectures composed of encoders and decoders are very popular for segmentation of medical images, satellite images etc. Typically, neural network initialized with weights from a network pre-trained on a large data set like ImageNet shows better performance than those trained from scratch on a small dataset. In some practical applications, particularly in medicine and traffic safety, the accuracy of the models is of utmost importance. In this paper, we demonstrate how the U-Net type architecture can be improved by the use of the pre-trained encoder. Our code and corresponding pre-trained weights are publicly available at https://github.com/ternaus/TernausNet. We compare three weight initialization schemes: LeCun uniform, the encoder with weights from VGG11 and full network trained on the Carvana dataset. This network architecture was a part of the winning solution (1st out of 735) in the Kaggle: Carvana Image Masking Challenge.

arXiv comment5 pages, 4 figures

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Satellite Imagery Feature Detection using Deep Convolutional Neural Network: A Kaggle Competition
• U-Net: Convolutional Networks for Biomedical Image Segmentation

Cited by in corpus (73)

• CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation
• Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning
• Global Guidance Network for Breast Lesion Segmentation in Ultrasound Images
• Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
• MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning
• Rethinking CNN Models for Audio Classification
• Deblending and Classifying Astronomical Sources with Mask R-CNN Deep Learning
• A Shallow U-Net Architecture for Reliably Predicting Blood Pressure (BP) from Photoplethysmogram (PPG) and Electrocardiogram (ECG) Signals
• SpaceNet MVOI: a Multi-View Overhead Imagery Dataset
• DeepFaceLab: Integrated, flexible and extensible face-swapping framework
• Angiodysplasia Detection and Localization Using Deep Convolutional Neural Networks
• Towards Better Surgical Instrument Segmentation in Endoscopic Vision: Multi-Angle Feature Aggregation and Contour Supervision
• 2017 Robotic Instrument Segmentation Challenge
• Overcoming Small Minirhizotron Datasets Using Transfer Learning
• Cross-dimensional transfer learning in medical image segmentation with deep learning
• Combining Background Subtraction Algorithms with Convolutional Neural Network
• Exploring Overcomplete Representations for Single Image Deraining using CNNs
• AP-MTL: Attention Pruned Multi-task Learning Model for Real-time Instrument Detection and Segmentation in Robot-assisted Surgery
• Robust Medical Instrument Segmentation Challenge 2019
• ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks
• Multi-frame Feature Aggregation for Real-time Instrument Segmentation in Endoscopic Video
• Automated Cardiothoracic Ratio Calculation and Cardiomegaly Detection using Deep Learning Approach
• RAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments
• Automatic salt deposits segmentation: A deep learning approach
• DeepFlash: Turning a Flash Selfie into a Studio Portrait
• Theoretical analysis and experimental validation of volume bias of soft Dice optimized segmentation maps in the context of inherent uncertainty
• Joint Iris Segmentation and Localization Using Deep Multi-task Learning Framework
• U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument
• Simple U-net Based Synthetic Polyp Image Generation: Polyp to Negative and Negative to Polyp
• Segmentation of Liver Lesions with Reduced Complexity Deep Models
• Deep-Learning Ensembles for Skin-Lesion Segmentation, Analysis, Classification: RECOD Titans at ISIC Challenge 2018
• SpaceNet 6: Multi-Sensor All Weather Mapping Dataset
• CF2-Net: Coarse-to-Fine Fusion Convolutional Network for Breast Ultrasound Image Segmentation
• Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge
• RASNet: Segmentation for Tracking Surgical Instruments in Surgical Videos Using Refined Attention Segmentation Network
• Unsupervised Medical Image Segmentation with Adversarial Networks: From Edge Diagrams to Segmentation Maps
• Crack Semantic Segmentation using the U-Net with Full Attention Strategy
• PECI-Net: Bolus segmentation from video fluoroscopic swallowing study images using preprocessing ensemble and cascaded inference
• Deep Landscape Features for Improving Vector-borne Disease Prediction
• Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networks
• RarePlanes: Synthetic Data Takes Flight
• Randomized Histogram Matching: A Simple Augmentation for Unsupervised Domain Adaptation in Overhead Imagery
• Interpretable and synergistic deep learning for visual explanation and statistical estimations of segmentation of disease features from medical images
• Effect of the output activation function on the probabilities and errors in medical image segmentation
• BARNet: Bilinear Attention Network with Adaptive Receptive Fields for Surgical Instrument Segmentation
• Deep Learning-based Aerial Image Segmentation with Open Data for Disaster Impact Assessment
• LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images
• TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors
• Few-Shot Defect Segmentation Leveraging Abundant Normal Training Samples Through Normal Background Regularization and Crop-and-Paste Operation
• Deepfake Image Generation for Improved Brain Tumor Segmentation
• CAggNet: Crossing Aggregation Network for Medical Image Segmentation
• Applying Knowledge Transfer for Water Body Segmentation in Peru
• Farmland Parcel Delineation Using Spatio-temporal Convolutional Networks
• BLESER: Bug Localization Based on Enhanced Semantic Retrieval
• Switching Loss for Generalized Nucleus Detection in Histopathology
• Out-of-distribution data supervision towards biomedical semantic segmentation
• Human Driver Behavior Prediction based on UrbanFlow
• Solving Traffic4Cast Competition with U-Net and Temporal Domain Adaptation
• Adversarial Policy Gradient for Deep Learning Image Augmentation
• A Study of Domain Generalization on Ultrasound-based Multi-Class Segmentation of Arteries, Veins, Ligaments, and Nerves Using Transfer Learning
• MSDU-net: A Multi-Scale Dilated U-net for Blur Detection
• Transfer Learning in Visual and Relational Reasoning
• Data-Free Adversarial Perturbations for Practical Black-Box Attack
• IntrinSeqNet: Learning to Estimate the Reflectance from Varying Illumination
• Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments
• Automatic Polyp Segmentation using U-Net-ResNet50
• A Survey on Green Deep Learning
• Towards Comparative Physical Interpretation of Spatial Variability Aware Neural Networks: A Summary of Results
• Improving Building Segmentation for Off-Nadir Satellite Imagery
• Customized OCT images compression scheme with deep neural network
• Using constraint structure and an improved object detection network to detect the 12^{th} Vertebra from CT images with a limited field of view for image-guided radiotherapy
• Network-Agnostic Knowledge Transfer for Medical Image Segmentation
• U-net super-neural segmentation and similarity calculation to realize vegetation change assessment in satellite imagery