6 papers
Implicit bias produces neural scaling laws in learning curves, from perceptrons to deep networks
Francesco D'Amico, Dario Bocchi, Matteo Negri
Scaling laws in deep learning -- empirical power-law relationships linking model performance to resource growth -- have emerged as simple yet striking regularities across architect…
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2
When do language diffusion models memorize their training data, and how to quantitatively assess their true generative regime? We address these questions by showing that Uniform-ba…
Memorization to Generalization: Emergence of Diffusion Models from Associative Memory
Bao Pham, Gabriel Raya, Matteo Negri +3
Dense Associative Memories (DenseAMs) are generalizations of Hopfield networks, which have superior information storage capacity and can store training data points (memories) at lo…
Benchmarking Graph Neural Networks in Solving Hard Constraint Satisfaction Problems
Geri Skenderi, Lorenzo Buffoni, Francesco D'Amico +6
Graph neural networks (GNNs) are increasingly applied to hard optimization problems, often claiming superiority over classical heuristics. However, such claims risk being unsolid d…
Statistical mechanics of vector Hopfield network near and above saturation
Flavio Nicoletti, Francesco D'Amico, Matteo Negri
We study analytically and numerically a Hopfield fully-connected network with -dimensional vector spins. These networks are models of associative memory that generalize the stan…
Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplings
Francesco D'Amico, Dario Bocchi, Luca Maria Del Bono +2
Energy-based probabilistic models learned by maximizing the likelihood of the data are limited by the intractability of the partition function. A widely used workaround is to maxim…