Self-Averaging and On-line Learning

Abstract

Conditions are given under which one may prove that the stochastic dynamics of on-line learning can be described by the deterministic evolution of a finite set of order parameters in the thermodynamic limit. A global constraint on the average magnitude of the increments in the stochastic process is necessary to ensure self-averaging. In the absence of such a constraint, convergence may only be in probability.

10 pages

Cited by in corpus (14)

• Mutual learning in a tree parity machine and its application to cryptography
• Cryptography based on neural networks - analytical results
• Opinion Dynamics of Learning Agents: Does Seeking Consensus Lead to Disagreement?
• Online Learning with Ensembles
• Prototype-based classifiers in the presence of concept drift: A modelling framework
• Supervised Learning in the Presence of Concept Drift: A modelling framework
• Generation of unpredictable time series by a Neural Network
• Neural Synchronization and Cryptography
• On-line learning dynamics of ReLU neural networks using statistical physics techniques
• Soft Mode in the Dynamics of Over-realizable On-line Learning for Soft Committee Machines
• Neural Networks, Game Theory and Time Series Generation
• Strategies for an efficient official publicity campaign
• Dynamics of Meta-learning Representation in the Teacher-student Scenario
• Statistical Mechanics of Node-perturbation Learning with Noisy Baseline