Weighted Sigmoid Gate Unit for an Activation Function of Deep Neural Network

Masayuki Tanaka

An activation function has crucial role in a deep neural network. A simple rectified linear unit (ReLU) are widely used for the activation function. In this paper, a weighted sigmo…