Analytical and Numerical Study of Internal Representations in Multilayer Neural Networks with Binary Weights

Abstract

We study the weight space structure of the parity machine with binary weights by deriving the distribution of volumes associated to the internal representations of the learning examples. The learning behaviour and the symmetry breaking transition are analyzed and the results are found to be in very good agreement with extended numerical simulations.

revtex, 20 pages + 9 figures, to appear in Phys. Rev. E

References in corpus (2)

• Weight Space Structure and Internal Representations: a Direct Approach to Learning and Generalization in Multilayer Neural Network
• Correlation of internal representations in feed-forward neural networks

Cited by in corpus (10)

• Learning by message-passing in networks of discrete synapses
• Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
• On the role of synaptic stochasticity in training low-precision neural networks
• Techniques of replica symmetry breaking and the storage problem of the McCulloch-Pitts neuron
• Weight space structure and analysis using a finite replica number in the Ising perceptron
• On the VC-dimension of neural networks with binary weights
• Multifractality and percolation in the coupling space of perceptrons
• Theory and learning protocols for the material tempotron model
• A multifractal phase-space analysis of perceptrons with biased patterns
• Multifractal analysis of perceptron learning with errors