Learning Activation Functions to Improve Deep Neural Networks

Forest Agostinelli, Matthew Hoffman, Peter Sadowski +1

Artificial neural networks typically have a fixed, non-linear activation function at each neuron. We have designed a novel form of piecewise linear activation function that is lear…