On the Depth of Monotone ReLU Neural Networks and ICNNs

Egor Bakaev, Florestan Brunck, Christoph Hertrich +2

We study two models of ReLU neural networks: monotone networks (ReLU${}^{+}$) and input convex neural networks (ICNN). Our focus is on expressivity, mostly in terms of depth, and we pr…