Phase Transitions of Neural Networks

Abstract

The cooperative behaviour of interacting neurons and synapses is studied using models and methods from statistical physics. The competition between training error and entropy may lead to discontinuous properties of the neural network. This is demonstrated for a few examples: Perceptron, associative memory, learning from examples, generalization, multilayer networks, structure recognition, Bayesian estimate, on-line training, noise estimation and time series generation.

Plenary talk for MINERVA workshop on mesoscopics, fractals and neural networks, Eilat, March 1997 Postscript File

Cited by in corpus (5)

• Mutual learning in a tree parity machine and its application to cryptography
• Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation
• Training a perceptron in a discrete weight space
• The Cavity Approach to Noisy Learning in Nonlinear Perceptrons
• Love might be a second-order phase transition