24 citations · 163 across the 60 of their papers we have counts for
7 papers · 1 filter
Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learning
Blake Bordelon, Cengiz Pehlevan
We provide an overview of high dimensional dynamical systems driven by random matrices, focusing on applications to simple models of learning and generalization in machine learning…
A note on the dynamics of extended-context disordered kinetic spin models
Jacob A. Zavatone-Veth, Cengiz Pehlevan
Inspired by striking advances in language modeling, there has recently been much interest in developing autogressive sequence models that are amenable to analytical study. In this…
Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
Alexander Atanasov, Blake Bordelon, Jacob A. Zavatone-Veth +2
We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide va…
Nadaraya-Watson kernel smoothing as a random energy model
Jacob A. Zavatone-Veth, Cengiz Pehlevan
Precise asymptotics have revealed many surprises in high-dimensional regression. These advances, however, have not extended to perhaps the simplest estimator: direct Nadaraya-Watso…
Replica method for eigenvalues of real Wishart product matrices
Jacob A. Zavatone-Veth, Cengiz Pehlevan
We show how the replica method can be used to compute the asymptotic eigenvalue spectrum of a real Wishart product matrix. For unstructured factors, this provides a compact, elemen…
On neural network kernels and the storage capacity problem
Jacob A. Zavatone-Veth, Cengiz Pehlevan
In this short note, we reify the connection between work on the storage capacity problem in wide two-layer treelike neural networks and the rapidly-growing body of literature on ke…