2 papers
cs.LG2025
The Resurrection of the ReLU
CoÅku Can Horuz, Geoffrey Kasenbacher, Saya Higuchi +7
Modeling sophisticated activation functions within deep learning architectures has evolved into a distinct research direction. Functions such as GELU, SELU, and SiLU offer smooth g…
cs.NE2024
Balanced Resonate-and-Fire Neurons
Saya Higuchi, Sebastian Kairat, Sander M. Bohte +1
The resonate-and-fire (RF) neuron, introduced over two decades ago, is a simple, efficient, yet biologically plausible spiking neuron model, which can extract frequency patterns wi…