U-Net Training with Instance-Layer Normalization
arXiv:1908.08466
Abstract
Normalization layers are essential in a Deep Convolutional Neural Network (DCNN). Various normalization methods have been proposed. The statistics used to normalize the feature maps can be computed at batch, channel, or instance level. However, in most of existing methods, the normalization for each layer is fixed. Batch-Instance Normalization (BIN) is one of the first proposed methods that combines two different normalization methods and achieve diverse normalization for different layers. However, two potential issues exist in BIN: first, the Clip function is not differentiable at input values of 0 and 1; second, the combined feature map is not with a normalized distribution which is harmful for signal propagation in DCNN. In this paper, an Instance-Layer Normalization (ILN) layer is proposed by using the Sigmoid function for the feature map combination, and cascading group normalization. The performance of ILN is validated on image segmentation of the Right Ventricle (RV) and Left Ventricle (LV) using U-Net as the network architecture. The results show that the proposed ILN outperforms previous traditional and popular normalization methods with noticeable accuracy improvements for most validations, supporting the effectiveness of the proposed ILN.
arXiv comment8 pages, 3 figures, accepted by MICCAI-MMMI 2019 workshop
References in corpus (6)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches
- A Real-time and Registration-free Framework for Dynamic Shape Instantiation
- Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units
- ACNN: a Full Resolution DCNN for Medical Image Segmentation