3 papers
cs.LG2026
OptiRoulette Optimizer: A New Stochastic Meta-Optimizer for up to 5.3x Faster Convergence
Stamatis Mastromichalakis
This paper presents OptiRoulette, a stochastic meta-optimizer that selects update rules during training instead of fixing a single optimizer. The method combines warmup optimizer l…
cs.LG2023
Parametric Leaky Tanh: A New Hybrid Activation Function for Deep Learning
Stamatis Mastromichalakis
Activation functions (AFs) are crucial components of deep neural networks (DNNs), having a significant impact on their performance. An activation function in a DNN is typically a s…
cs.LG2020
ALReLU: A different approach on Leaky ReLU activation function to improve Neural Networks Performance
Stamatis Mastromichalakis
Despite the unresolved 'dying ReLU problem', the classical ReLU activation function (AF) has been extensively applied in Deep Neural Networks (DNN), in particular Convolutional Neu…