Inference and learning in sparse systems with multiple states

Abstract

We discuss how inference can be performed when data are sampled from the non-ergodic phase of systems with multiple attractors. We take as model system the finite connectivity Hopfield model in the memory phase and suggest a cavity method approach to reconstruct the couplings when the data are separately sampled from few attractor states. We also show how the inference results can be converted into a learning protocol for neural networks in which patterns are presented through weak external fields. The protocol is simple and fully local, and is able to store patterns with a finite overlap with the input patterns without ever reaching a spin glass phase where all memories are lost.

15 pages, 10 figures, to be published in Phys. Rev. E

References in corpus (7)

• Statistical physics of pairwise probability models
• Small-correlation expansions for the inverse Ising problem
• Mean Field Theory For Non-Equilibrium Network Reconstruction
• Efficient supervised learning in networks with binary synapses
• Intrinsic limitations of inverse inference in the pairwise Ising spin glass
• Dynamics and Performance of Susceptibility Propagation on Synthetic Data
• Reconstructing the Hopfield network as an inverse Ising problem

Cited by in corpus (10)

• Inverse statistical problems: from the inverse Ising problem to data science
• Large deviations of cascade processes on graphs
• Mean-field theory for the inverse Ising problem at low temperatures
• Inference of kinetic Ising model on sparse graphs
• Solving the inverse Ising problem by mean-field methods in a clustered phase space with many states
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Clustering of neural codewords revealed by a first-order phase transition
• Daydreaming Hopfield Networks and their surprising effectiveness on correlated data
• From statistical inference to a differential learning rule for stochastic neural networks
• Sparse Hopfield network reconstruction with ${\ell }_1$ regularization