Generalized Activation via Multivariate Projection

Jiayun Li, Yuxiao Cheng, Yiwen Lu +3

Activation functions are essential to introduce nonlinearity into neural networks, with the Rectified Linear Unit (ReLU) often favored for its simplicity and effectiveness. Motivat…