Neural Networks Trained by Weight Permutation are Universal Approximators

Yongqiang Cai, Gaohang Chen, Zhonghua Qiao

The universal approximation property is fundamental to the success of neural networks, and has traditionally been achieved by training networks without any constraints on their par…