Universal approximation and model compression for radial neural networks

Iordan Ganev, Twan van Laarhoven, Robin Walters

We introduce a class of fully-connected neural networks whose activation functions, rather than being pointwise, rescale feature vectors by a function depending only on their norm.…