A Differentiable Transition Between Additive and Multiplicative Neurons

Wiebke Köpp, Patrick van der Smagt, Sebastian Urban

Existing approaches to combine both additive and multiplicative neural units either use a fixed assignment of operations or require discrete optimization to determine what function…