Evolutionary Optimization of Deep Learning Activation Functions

Garrett Bingham, William Macke, Risto Miikkulainen

The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, t…