Injectivity of ReLU networks: perspectives from statistical physics

Antoine Maillard, Afonso S. Bandeira, David Belius +2

When can the input of a ReLU neural network be inferred from its output? In other words, when is the network injective? We consider a single layer, $x\mapsto \mathrm{ReLU}(Wx)$, w…