Representing MAX functions using two-hidden-layer ReLU networks

Zhimao Wang, Amitabh Basu

We study exact representations of ${\mathrm{MAX}}_N(x)=\max x_1,\dots ,x_N$ using two-hidden-layer ReLU neural networks. This problem has been studied in recent years in an attempt…