Expressivity Saturation: Reduced Affine Region Usage Under Increasing Task Complexity

Xuan Qi, Yi Wei, Fanqi Yu +1

Piecewise-affine neural networks (e.g., with ReLU or LeakyReLU activations) implement continuous piecewise-affine maps, and the number of affine regions provides a natural proxy fo…