Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes

Christian Haase, Christoph Hertrich, Georg Loho

We prove that the set of functions representable by ReLU neural networks with integer weights strictly increases with the network depth while allowing arbitrary width. More precise…