Normalization in Training U-Net for 2D Biomedical Semantic Segmentation
arXiv:1809.03783
Abstract
2D biomedical semantic segmentation is important for robotic vision in surgery. Segmentation methods based on Deep Convolutional Neural Network (DCNN) can out-perform conventional methods in terms of both accuracy and levels of automation. One common issue in training a DCNN for biomedical semantic segmentation is the internal covariate shift where the training of convolutional kernels is encumbered by the distribution change of input features, hence both the training speed and performance are decreased. Batch Normalization (BN) is the first proposed method for addressing internal covariate shift and is widely used. Instance Normalization (IN) and Layer Normalization (LN) have also been proposed. Group Normalization (GN) is proposed more recently and has not yet been applied to 2D biomedical semantic segmentation, however, no specific validations on GN were given. Most DCNNs for biomedical semantic segmentation adopt BN as the normalization method by default, without reviewing its performance. In this paper, four normalization methods - BN, IN, LN and GN are compared in details, specifically for 2D biomedical semantic segmentation. U-Net is adopted as the basic DCNN structure. Three datasets regarding the Right Ventricle (RV), aorta, and Left Ventricle (LV) are used for the validation. The results show that detailed subdivision of the feature map, i.e. GN with a large group number or IN, achieves higher accuracy. This accuracy improvement mainly comes from better model generalization. Codes are uploaded and maintained at Xiao-Yun Zhou's Github.
arXiv comment5 figures, 5 tables, published on IEEE Robotics and Automation Letters 2019
References in corpus (7)
- A Survey on Deep Learning in Medical Image Analysis
- Instance Normalization: The Missing Ingredient for Fast Stylization
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
- Don't Decay the Learning Rate, Increase the Batch Size
- Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks
- Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks