5 papers
The Resurrection of the ReLU
CoÅku Can Horuz, Geoffrey Kasenbacher, Saya Higuchi +7
Modeling sophisticated activation functions within deep learning architectures has evolved into a distinct research direction. Functions such as GELU, SELU, and SiLU offer smooth g…
Convolutional Neural Networks Do Work with Pre-Defined Filters
Christoph Linse, Erhardt Barth, Thomas Martinetz
We present a novel class of Convolutional Neural Networks called Pre-defined Filter Convolutional Neural Networks (PFCNNs), where all nxn convolution kernels with n>1 are pre-defin…
Leaky ReLUs That Differ in Forward and Backward Pass Facilitate Activation Maximization in Deep Neural Networks
Christoph Linse, Erhardt Barth, Thomas Martinetz
Activation maximization (AM) strives to generate optimal input stimuli, revealing features that trigger high responses in trained deep neural networks. AM is an important method of…
Enhancing Generalization in Convolutional Neural Networks through Regularization with Edge and Line Features
Christoph Linse, Beatrice Brückner, Thomas Martinetz
This paper proposes a novel regularization approach to bias Convolutional Neural Networks (CNNs) toward utilizing edge and line features in their hidden layers. Rather than learnin…
Rethinking generalization of classifiers in separable classes scenarios and over-parameterized regimes
Julius Martinetz, Christoph Linse, Thomas Martinetz
We investigate the learning dynamics of classifiers in scenarios where classes are separable or classifiers are over-parameterized. In both cases, Empirical Risk Minimization (ERM)…