Activation Adaptation in Neural Networks

Farnoush Farhadi, Vahid Partovi Nia, Andrea Lodi

Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias an…