1.2k citations · 2.2k across the 46 of their papers we have counts for
4 papers · 1 filter
Demystifying Spectral Bias on Real-World Data
Itay Lavie, Zohar Ringel
Kernel ridge regression (KRR) and Gaussian processes (GPs) are fundamental tools in statistics and machine learning, with recent applications to highly over-parameterized deep neur…
Critical feature learning in deep neural networks
Kirsten Fischer, Javed Lindner, David Dahmen +3
A key property of neural networks driving their success is their ability to learn features from data. Understanding feature learning from a theoretical viewpoint is an emerging fie…
Wilsonian Renormalization of Neural Network Gaussian Processes
Jessica N. Howard, Ro Jefferson, Anindita Maiti +1
Separating relevant and irrelevant information is key to any modeling process or scientific inquiry. Theoretical physics offers a powerful tool for achieving this in the form of th…
Towards Understanding Inductive Bias in Transformers: A View From Infinity
Itay Lavie, Guy Gur-Ari, Zohar Ringel
We study inductive bias in Transformers in the infinitely over-parameterized Gaussian process limit and argue transformers tend to be biased towards more permutation symmetric func…