Breaking the Conventional Forward-Backward Tie in Neural Networks: Activation Functions

Luigi Troiano, Francesco Gissi, Vincenzo Benedetto +1

Gradient-based neural network training traditionally enforces symmetry between forward and backward propagation, requiring activation functions to be differentiable (or sub-differe…