The Geometric Structure of Fully-Connected ReLU Layers

Jonatan Vallin, Karl Larsson, Mats G. Larson

We formalize and interpret the geometric structure of $d$-dimensional fully connected ReLU layers in neural networks. The parameters of a ReLU layer induce a natural partition of t…