On Symmetry and Initialization for Neural Networks

Ido Nachum, Amir Yehudayoff

This work provides an additional step in the theoretical understanding of neural networks. We consider neural networks with one hidden layer and show that when learning symmetric f…