A Neural Transfer Function for a Smooth and Differentiable Transition Between Additive and Multiplicative Interactions

Sebastian Urban, Patrick van der Smagt

Existing approaches to combine both additive and multiplicative neural units either use a fixed assignment of operations or require discrete optimization to determine what function…