Why Rectified Power Unit Networks Fail and How to Improve It: An Effective Field Theory Perspective

Taeyoung Kim, Myungjoo Kang

The Rectified Power Unit (RePU) activation function, a differentiable generalization of the Rectified Linear Unit (ReLU), has shown promise in constructing neural networks due to i…