The Geometry of the Set of Equivalent Linear Neural Networks

Jonathan Richard Shewchuk, Sagnik Bhattacharya

We characterize the geometry of the set of all weight vectors for which a fully-connected linear neural network computes the same linear transformation $W$. Said another way, the s…