On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth

Gennadiy Averkov, Christopher Hojny, Maximilian Merkert

To confirm that the expressive power of ReLU neural networks grows with their depth, the function $F_n=\max \{0,x_1,\dots ,x_n\}$ has been considered in the literature. A conject…