Multilayer neural networks with extensively many hidden units

Abstract

The information processing abilities of a multilayer neural network with a number of hidden units scaling as the input dimension are studied using statistical mechanics methods. The mapping from the input layer to the hidden units is performed by general symmetric Boolean functions whereas the hidden layer is connected to the output by either discrete or continuous couplings. Introducing an overlap in the space of Boolean functions as order parameter the storage capacity if found to scale with the logarithm of the number of implementable Boolean functions. The generalization behaviour is smooth for continuous couplings and shows a discontinuous transition to perfect generalization for discrete ones.

4 pages, 2 figures

Cited by in corpus (7)

• Perturbation propagation in random and evolved Boolean networks
• Noise in random Boolean networks
• Boolean networks with reliable dynamics
• Attractor and Basin Entropies of Random Boolean Networks Under Asynchronous Stochastic Update
• Response of Boolean networks to perturbations
• Single-Nodal Spontaneous Symmetry Breaking in NLP Models
• No one-hidden-layer neural network can represent multivariable functions