Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?

Nitin Bansal, Xiaohan Chen, Zhangyang Wang

This paper seeks to answer the question: as the (near-) orthogonality of weights is found to be a favorable property for training deep convolutional neural networks, how can we enf…