Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descent

Max Hennick, Stijn De Baerdemacker

We show that the behavior of stochastic gradient descent is related to Bayesian statistics by showing that SGD is effectively diffusion on a fractal landscape, where the fractal di…