Neural Networks With Dense Weights Are Not Universal Approximators

Levi Rauchwerger, Stefanie Jegelka, Ron Levie

We investigate the approximation capabilities of dense neural networks. While universal approximation theorems establish that sufficiently large architectures can approximate arbit…