TAOTF: A Two-stage Approximately Orthogonal Training Framework in Deep Neural Networks

Taoyong Cui, Jianze Li, Yuhan Dong +1

The orthogonality constraints, including the hard and soft ones, have been used to normalize the weight matrices of Deep Neural Network (DNN) models, especially the Convolutional N…