Neural networks with trainable matrix activation functions

Zhengqi Liu, Shuhao Cao, Yuwen Li +1

The training process of neural networks usually optimize weights and bias parameters of linear transformations, while nonlinear activation functions are pre-specified and fixed. Th…