Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units

Yixi Xu, Xiao Wang

This paper presents a general framework for norm-based capacity control for $L_{p,q}$ weight normalized deep neural networks. We establish the upper bound on the Rademacher complex…