4 papers
An exact information theory of generalization phase transitions in Bayesian diffusion models
Henry Hunt, Mason Kamb, Surya Ganguli
How diffusion models circumvent the curse of dimensionality to learn complex distributions over high dimensional spaces from a finite training set, instead of memorizing it, remain…
Deriving Neural Scaling Laws from the statistics of natural language
Francesco Cagnetta, Allan Raventós, Surya Ganguli +1
Despite the fact that experimental neural scaling laws have substantially guided empirical progress in large-scale machine learning, no existing theory can quantitatively predict t…
From Kepler to Newton: Inductive Biases Guide Learned World Models in Transformers
Ziming Liu, Sophia Sanborn, Surya Ganguli +1
Can general-purpose AI architectures go beyond prediction to discover the physical laws governing the universe? True intelligence relies on "world models" -- causal abstractions th…
High-capacity associative memory in a quantum-optical spin glass
Brendan P. Marsh, David Atri Schuller, Yunpeng Ji +5
The Hopfield model describes a neural network that stores memories using all-to-all-coupled spins. Memory patterns are recalled under equilibrium dynamics. Storing too many pattern…