Spontaneous Kolmogorov-Arnold Geometry in Shallow MLPs

Michael H. Freedman, Michael Mulligan

The Kolmogorov-Arnold (KA) representation theorem constructs universal, but highly non-smooth inner functions (the first layer map) in a single (non-linear) hidden layer neural net…