260 citations · 687 across the 25 of their papers we have counts for
8 papers · 1 filter
Normalization Techniques in Training DNNs: Methodology, Analysis and Application
Lei Huang, Jie Qin, Yi Zhou +3
Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various app…
Group Whitening: Balancing Learning Efficiency and Representational Capacity
Lei Huang, Yi Zhou, Li Liu +2
Batch normalization (BN) is an important technique commonly incorporated into deep learning models to perform standardization within mini-batches. The merits of BN in improving a m…
Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition
Ionut Cosmin Duta, Li Liu, Fan Zhu +1
This work introduces pyramidal convolution (PyConv), which is capable of processing the input at multiple filter scales. PyConv contains a pyramid of kernels, where each level invo…
Improved Residual Networks for Image and Video Recognition
Ionut Cosmin Duta, Li Liu, Fan Zhu +1
Residual networks (ResNets) represent a powerful type of convolutional neural network (CNN) architecture, widely adopted and used in various tasks. In this work we propose an impro…
Controllable Orthogonalization in Training DNNs
Lei Huang, Li Liu, Fan Zhu +4
Orthogonality is widely used for training deep neural networks (DNNs) due to its ability to maintain all singular values of the Jacobian close to 1 and reduce redundancy in represe…
An Investigation into the Stochasticity of Batch Whitening
Lei Huang, Lei Zhao, Yi Zhou +3
Batch Normalization (BN) is extensively employed in various network architectures by performing standardization within mini-batches. A full understanding of the process has been a…