3 papers
cs.LG2025
The Resurrection of the ReLU
Coşku Can Horuz, Geoffrey Kasenbacher, Saya Higuchi +7
Modeling sophisticated activation functions within deep learning architectures has evolved into a distinct research direction. Functions such as GELU, SELU, and SiLU offer smooth g…
cs.NE2024
Understanding the Convergence in Balanced Resonate-and-Fire Neurons
Saya Higuchi, Sander M. Bohte, Sebastian Otte
Resonate-and-Fire (RF) neurons are an interesting complementary model for integrator neurons in spiking neural networks (SNNs). Due to their resonating membrane dynamics they can e…
cs.NE2024
Balanced Resonate-and-Fire Neurons
Saya Higuchi, Sebastian Kairat, Sander M. Bohte +1
The resonate-and-fire (RF) neuron, introduced over two decades ago, is a simple, efficient, yet biologically plausible spiking neuron model, which can extract frequency patterns wi…