Pure and Spurious Critical Points: a Geometric Study of Linear Networks

Matthew Trager, Kathlén Kohn, Joan Bruna

The critical locus of the loss function of a neural network is determined by the geometry of the functional space and by the parameterization of this space by the network's weights…