3 papers
cs.LG2025
The Resurrection of the ReLU
Coşku Can Horuz, Geoffrey Kasenbacher, Saya Higuchi +7
Modeling sophisticated activation functions within deep learning architectures has evolved into a distinct research direction. Functions such as GELU, SELU, and SiLU offer smooth g…
cs.LG2025
On Space Folds of ReLU Neural Networks
Michal Lewandowski, Hamid Eghbalzadeh, Bernhard Heinzl +2
Recent findings suggest that the consecutive layers of ReLU neural networks can be understood geometrically as space folding transformations of the input space, revealing patterns…
cs.NE2024
CantorNet: A Sandbox for Testing Geometrical and Topological Complexity Measures
Michal Lewandowski, Hamid Eghbalzadeh, Bernhard A. Moser
Many natural phenomena are characterized by self-similarity, for example the symmetry of human faces, or a repetitive motif of a song. Studying of such symmetries will allow us to…