Fractional Optimizers Meet Fractal Activation Functions: An Empirical Study of Multi-Scale Optimization in Neural Network

Sebastian Raubitzek, Georg Goldenits, Sebastian Schrittwieser +2

Fractional optimization methods and fractal activation functions are two independent directions for improving neural network training. Fractional optimizers extend first-order opti…