3 citations · 5 across the 12 of their papers we have counts for
6 papers · 1 filter
Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis +1
Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a…
xRFM: Accurate, scalable, and interpretable feature learning models for tabular data
Daniel Beaglehole, David Holzmüller, Adityanarayanan Radhakrishnan +1
Inference from tabular data, collections of continuous and categorical variables organized into matrices, is a foundation for modern technology and science. Yet, in contrast to the…
A Gap Between the Gaussian RKHS and Neural Networks: An Infinite-Center Asymptotic Analysis
Akash Kumar, Rahul Parhi, Mikhail Belkin
Recent works have characterized the function-space inductive bias of infinite-width bounded-norm single-hidden-layer neural networks as a kind of bounded-variation-type space. This…
Mirror Descent on Reproducing Kernel Banach Spaces
Akash Kumar, Mikhail Belkin, Parthe Pandit
Recent advances in machine learning have led to increased interest in reproducing kernel Banach spaces (RKBS) as a more general framework that extends beyond reproducing kernel Hil…
Context-Scaling versus Task-Scaling in In-Context Learning
Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1
Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…
Average gradient outer product as a mechanism for deep neural collapse
Daniel Beaglehole, Peter Súkeník, Marco Mondelli +1
Deep Neural Collapse (DNC) refers to the surprisingly rigid structure of the data representations in the final layers of Deep Neural Networks (DNNs). Though the phenomenon has been…