SwishReLU: A Unified Approach to Activation Functions for Enhanced Deep Neural Networks Performance

Jamshaid Ul Rahman, Rubiqa Zulfiqar, Asad Khan +1

ReLU, a commonly used activation function in deep neural networks, is prone to the issue of "Dying ReLU". Several enhanced versions, such as ELU, SeLU, and Swish, have been introdu…