Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
The Resurrection of the ReLU
Coşku Can Horuz, Geoffrey Kasenbacher, Saya Higuchi +7
Modeling sophisticated activation functions within deep learning architectures has evolved into a distinct research direction. Functions such as GELU, SELU, and SiLU offer smooth g…
cs.LG2025
On Space Folds of ReLU Neural Networks
Michal Lewandowski, Hamid Eghbalzadeh, Bernhard Heinzl +2
Recent findings suggest that the consecutive layers of ReLU neural networks can be understood geometrically as space folding transformations of the input space, revealing patterns…