Sharp Representation Theorems for ReLU Networks with Precise Dependence on Depth

Guy Bresler, Dheeraj Nagaraj

We prove sharp dimension-free representation results for neural networks with $D$ ReLU layers under square loss for a class of functions ${\mathcal{G}}_D$ defined in the paper. These…