Extensive load in multitasking associative networks

Abstract

We use belief-propagation techniques to study the equilibrium behavior of a bipartite spin-glass, with interactions between two sets of $N$ and $P=\alpha N$ spins. Each spin has a finite degree, i.e.\ number of interaction partners in the opposite set; an equivalent view is then of a system of $N$ neurons storing $P$ diluted patterns. We show that in a large part of the parameter space of noise, dilution and storage load, delimited by a critical surface, the network behaves as an extensive parallel processor, retrieving all $P$ patterns {\it in parallel} without falling into spurious states due to pattern cross-talk and typical of the structural glassiness built into the network. Our approach allows us to consider effects beyond those studied in replica theory so far, including pattern asymmetry and heterogeneous dilution. Parallel extensive retrieval is more robust for homogeneous degree distributions, and is not disrupted by biases in the distributions of the spin-glass links.

References in corpus (14)

• Hierarchical organization of modularity in metabolic networks
• Random graphs with arbitrary degree distributions and their applications
• Specificity and stability in topology of protein networks
• Critical phenomena in complex networks
• Universal Behavior of Load Distribution in Scale-free Networks
• The Bethe lattice spin glass revisited
• Classification of scale-free networks
• Multitasking associative networks
• Immune networks: multi-tasking capabilities near saturation
• A thermodynamical perspective of immune capabilities
• Intrinsic degree-correlations in static model of scale-free networks
• A Hebbian approach to complex network generation
• Finite size corrections to disordered systems on Erdös-Rényi random graphs
• Network resilience against intelligent attacks constrained by degree dependent node removal cost

Cited by in corpus (34)

• Emergence of Compositional Representations in Restricted Boltzmann Machines
• Regulation of Irregular Neuronal Firing by Autaptic Transmission
• Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
• Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors
• Frequency-difference-dependent stochastic resonance in neural systems
• Phase transitions in Restricted Boltzmann Machines with generic priors
• From Dyson to Hopfield: Processing on hierarchical networks
• Non-Convex Multi-species Hopfield models
• Neural Networks retrieving Boolean patterns in a sea of Gaussian ones
• Hierarchical neural networks perform both serial and parallel processing
• The ambiguity of nestedness under soft and hard constraints
• Correlated fluctuations in strongly-coupled binary networks beyond equilibrium
• Notes on stochastic (bio)-logic gates: the role of allosteric cooperativity
• A complete mean-field theory for dynamics of binary recurrent neural networks
• The average number of distinct sites visited by a random walker on random graphs
• Emerging heterogeneities in Italian customs and comparison with nearby countries
• Topological dimension tunes activity patterns in hierarchical modular network models
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Parallel Learning by Multitasking Neural Networks
• Dense Hopfield Networks in the Teacher-Student Setting
• A dynamical model of the adaptive immune system: effects of cells promiscuity, antigens and B-B interactions
• Role of zero synapses in unsupervised feature learning
• Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting
• The effect of priors on Learning with Restricted Boltzmann Machines
• Optimized brute-force algorithms for the bifurcation analysis of a spin-glass-like neural network model
• Vitality of Neural Networks under Reoccurring Catastrophic Failures
• The role of idiotypic interactions in the adaptive immune system: a belief-propagation approach
• Associative networks with diluted patterns: dynamical analysis at low and medium load
• Regular graphs maximize the variability of random neural networks
• A relativistic extension of Hopfield neural networks via the mechanical analogy
• Protein interaction networks and biology: towards the connection
• Overcoming the complexity barrier of the dynamic message-passing method in networks with fat-tailed degree distributions
• Response time of lateral predictive coding and benefits of modular structures
• Uncovering the non-equilibrium stationary properties in sparse Boolean networks