SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Training

Ildus Sadrtdinov, Ivan Klimov, Ekaterina Lobacheva +1

We present a thermodynamic interpretation of the stationary behavior of stochastic gradient descent (SGD) under fixed learning rates (LRs) in neural network training. We show that…