Stability of the replica symmetric solution for the information conveyed by by a neural network

Abstract

The information that a pattern of firing in the output layer of a feedforward network of threshold-linear neurons conveys about the network's inputs is considered. A replica-symmetric solution is found to be stable for all but small amounts of noise. The region of instability depends on the contribution of the threshold and the sparseness: for distributed pattern distributions, the unstable region extends to higher noise variances than for very sparse distributions, for which it is almost nonexistant.

arXiv comment19 pages, LaTeX, 5 figures. Also available at http://www.mrc-bbc.ox.ac.uk/~schultz/papers.html . Submitted to Phys. Rev. E Minor changes

Cited by in corpus (9)

• The mutual information of a stochastic binary channel: validity of the Replica Symmetry Ansatz
• Estimating Information-Theoretic Quantities
• Replica symmetric evaluation of the information transfer in a two-layer network in presence of continuous and discrete stimuli
• Numerical simulation of a binary communication channel: Comparison between a replica calculation and an exact solution
• A diagrammatic approach to study the information transfer in weakly non-linear channels
• Mutual Information of Three-State Low Activity Diluted Neural Networks with Self-Control
• Adaptive thresholds for neural networks with synaptic noise
• A replica free evaluation of the neuronal population information with mixed continuous and discrete stimuli: from the linear to the asymptotic regime
• Self-control dynamics for sparsely coded networks with synaptic noise