Optimization on Submanifolds of Convolution Kernels in CNNs

Mete Ozay, Takayuki Okatani

Kernel normalization methods have been employed to improve robustness of optimization methods to reparametrization of convolution kernels, covariate shift, and to accelerate traini…