Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth

Filipe de Avila Belbute-Peres, J. Zico Kolter

Neural networks with sinusoidal activations have been proposed as an alternative to networks with traditional activation functions. Despite their promise, particularly for learning…