106 citations · 317 across the 65 of their papers we have counts for
7 papers · 1 filter
The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes
Alexander Atanasov, Blake Bordelon, Sabarish Sainathan +1
For small training set sizes , the generalization error of wide neural networks is well-approximated by the error of an infinite width neural network (NN), either in the kernel…
The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks
Blake Bordelon, Cengiz Pehlevan
It is unclear how changing the learning rule of a deep neural network alters its learning dynamics and representations. To gain insight into the relationship between learned featur…
Replica method for eigenvalues of real Wishart product matrices
Jacob A. Zavatone-Veth, Cengiz Pehlevan
We show how the replica method can be used to compute the asymptotic eigenvalue spectrum of a real Wishart product matrix. For unstructured factors, this provides a compact, elemen…
Bandwidth Enables Generalization in Quantum Kernel Models
Abdulkadir Canatar, Evan Peters, Cengiz Pehlevan +2
Quantum computers are known to provide speedups over classical state-of-the-art machine learning methods in some specialized settings. For example, quantum kernel methods have been…
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks
Blake Bordelon, Cengiz Pehlevan
We analyze feature learning in infinite-width neural networks trained with gradient flow through a self-consistent dynamical field theory. We construct a collection of deterministi…
Contrasting random and learned features in deep Bayesian linear regression
Jacob A. Zavatone-Veth, William L. Tong, Cengiz Pehlevan
Understanding how feature learning affects generalization is among the foremost goals of modern deep learning theory. Here, we study how the ability to learn representations affect…
On neural network kernels and the storage capacity problem
Jacob A. Zavatone-Veth, Cengiz Pehlevan
In this short note, we reify the connection between work on the storage capacity problem in wide two-layer treelike neural networks and the rapidly-growing body of literature on ke…