Gardner formula for Ising perceptron models at small densities

Abstract

We consider the Ising perceptron model with N spins and M = N*alpha patterns, with a general activation function U that is bounded above. For U bounded away from zero, or U a one-sided threshold function, it was shown by Talagrand (2000, 2011) that for small densities alpha, the free energy of the model converges in the large-N limit to the replica symmetric formula conjectured in the physics literature (Krauth--Mezard 1989, see also Gardner--Derrida 1988). We give a new proof of this result, which covers the more general class of all functions U that are bounded above and satisfy a certain variance bound. The proof uses the (first and second) moment method conditional on the approximate message passing iterates of the model. In order to deduce our main theorem, we also prove a new concentration result for the perceptron model in the case where U is not bounded away from zero.

References in corpus (8)

• Message Passing Algorithms for Compressed Sensing
• Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
• Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems
• A Theory of Solving TAP Equations for Ising Models with General Invariant Random Matrices
• Memory-free dynamics for the TAP equations of Ising models with arbitrary rotation invariant ensembles of random coupling matrices
• Biased landscapes for random Constraint Satisfaction Problems
• The replica-symmetric free energy for Ising spin glasses with orthogonally invariant couplings
• Another look at the Gardner problem