Generalization in Learning: Eyring Formula and Predator-prey Model
arXiv:2610.35965 · doi:10.1134/S1995080225607970
Abstract
In this work, generalization in machine learning is investigated using a predator-prey model. We overview biological and chemical interpretation of the stochastic gradient Langevin dynamics and generative adversarial network. For a particular example of recovering an unknown function by a polynomial of fixed degree from a set of noisy data, the predator-prey model provides better approximation compared to the gradient descent method. Further, thermodynamical arguments, in particular Eyring formula, are used to explain grokking (delayed generalization) phenomenon in machine learning.
arXiv comment12 pages, 4 figures