Self-Averaging and On-line Learning
arXiv:cond-mat/9805339 · doi:10.1103/PhysRevLett.80.5445
Abstract
Conditions are given under which one may prove that the stochastic dynamics of on-line learning can be described by the deterministic evolution of a finite set of order parameters in the thermodynamic limit. A global constraint on the average magnitude of the increments in the stochastic process is necessary to ensure self-averaging. In the absence of such a constraint, convergence may only be in probability.
10 pages