3 citations · 5 across the 12 of their papers we have counts for
8 papers · 1 filter
Fast training of large kernel models with delayed projections
Amirhesam Abedsoltan, Siyuan Ma, Parthe Pandit +1
Classical kernel machines have historically faced significant challenges in scaling to large datasets and model sizes--a key ingredient that has driven the success of neural networ…
Mirror Descent on Reproducing Kernel Banach Spaces
Akash Kumar, Mikhail Belkin, Parthe Pandit
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hil…
Context-Scaling versus Task-Scaling in In-Context Learning
Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1
Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…
Eigenvectors of the De Bruijn Graph Laplacian: A Natural Basis for the Cut and Cycle Space
Anthony Philippakis, Neil Mallinar, Parthe Pandit +1
We study the Laplacian of the undirected De Bruijn graph over an alphabet of order . While the eigenvalues of this Laplacian were found in 1998 by Delorme and Tillich [1], a…
Emergence in non-neural models: grokking modular arithmetic via average gradient outer product
Neil Mallinar, Daniel Beaglehole, Libin Zhu +3
Neural networks trained to solve modular arithmetic tasks exhibit grokking, a phenomenon where the test accuracy starts improving long after the model achieves 100% training accura…
Average gradient outer product as a mechanism for deep neural collapse
Daniel Beaglehole, Peter Súkeník, Marco Mondelli +1
Deep Neural Collapse (DNC) refers to the surprisingly rigid structure of the data representations in the final layers of Deep Neural Networks (DNNs). Though the phenomenon has been…
On the Nystrom Approximation for Preconditioning in Kernel Machines
Amirhesam Abedsoltan, Parthe Pandit, Luis Rademacher +1
Kernel methods are a popular class of nonlinear predictive models in machine learning. Scalable algorithms for learning kernel models need to be iterative in nature, but convergenc…
Linear Recursive Feature Machines provably recover low-rank matrices
Adityanarayanan Radhakrishnan, Mikhail Belkin, Dmitriy Drusvyatskiy
A fundamental problem in machine learning is to understand how neural networks make accurate predictions, while seemingly bypassing the curse of dimensionality. A possible explanat…