Geometrical structures of digital fluctuations in parameter space of neural networks trained with adaptive momentum optimization

Igor V. Netay

We present results of numerical experiments for neural networks with stochastic gradient-based optimization with adaptive momentum. This widely applied optimization has proved conv…