Shallower ReLU Network Representations via Exact Linear Algebra

Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7

We prove that the maximum of $n$ real numbers is exactly representable by a ReLU network with two hidden layers for every $n\le 10$. The constructions are obtained by reducing the…