69 citations · 217 across the 26 of their papers we have counts for
16 papers · 1 filter
Exact Dynamics and Finite-Sample Trajectory Recovery of Linear Recursive Feature Machines
Andrew Cheng, Bobak T. Kiani, Yue M. Lu +1
Recursive feature machines (RFMs) learn representations of data by alternating between fitting a predictor to a dataset and updating features of that predictor using the average gr…
The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks
Taehun Cha, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1
Understanding how deep neural networks learn representations remains a central challenge in machine learning theory. In this work, we propose a feature-centric framework for analyz…
Efficient and accurate steering of Large Language Models through attention-guided feature learning
Parmida Davarmanesh, Ashia Wilson, Adityanarayanan Radhakrishnan
Steering, or direct manipulation of internal activations to guide LLM responses toward specific semantic concepts, is emerging as a promising avenue for both understanding how sema…
xRFM: Accurate, scalable, and interpretable feature learning models for tabular data
Daniel Beaglehole, David Holzmüller, Adityanarayanan Radhakrishnan +1
Inference from tabular data, collections of continuous and categorical variables organized into matrices, is a foundation for modern technology and science. Yet, in contrast to the…
Context-Scaling versus Task-Scaling in In-Context Learning
Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1
Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning
Libin Zhu, Chaoyue Liu, Adityanarayanan Radhakrishnan +1
In this paper, we first present an explanation regarding the common occurrence of spikes in the training loss when neural networks are trained with stochastic gradient descent (SGD…
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features
Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit +1
In recent years neural networks have achieved impressive results on many technological and scientific tasks. Yet, the mechanism through which these models automatically select feat…
Transfer Learning with Kernel Methods
Adityanarayanan Radhakrishnan, Max Ruiz Luyten, Neha Prasad +1
Transfer learning refers to the process of adapting a model trained on a source task to a target task. While kernel methods are conceptually and computationally simple machine lear…
Quadratic models for understanding catapult dynamics of neural networks
Libin Zhu, Chaoyue Liu, Adityanarayanan Radhakrishnan +1
While neural networks can be approximated by linear models as their width increases, certain properties of wide neural networks cannot be captured by linear models. In this work we…
Wide and Deep Neural Networks Achieve Optimality for Classification
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
While neural networks are used for classification tasks across domains, a long-standing open problem in machine learning is determining whether neural networks trained using standa…
Simple, Fast, and Flexible Framework for Matrix Completion with Infinite Width Neural Networks
Adityanarayanan Radhakrishnan, George Stefanakis, Mikhail Belkin +1
Matrix completion problems arise in many applications including recommendation systems, computer vision, and genomics. Increasingly larger neural networks have been successful in m…
A Mechanism for Producing Aligned Latent Spaces with Autoencoders
Saachi Jain, Adityanarayanan Radhakrishnan, Caroline Uhler
Aligned latent spaces, where meaningful semantic shifts in the input space correspond to a translation in the embedding space, play an important role in the success of downstream t…
Increasing Depth Leads to U-Shaped Test Risk in Over-parameterized Convolutional Networks
Eshaan Nichani, Adityanarayanan Radhakrishnan, Caroline Uhler
Recent works have demonstrated that increasing model capacity through width in over-parameterized neural networks leads to a decrease in test risk. For neural networks, however, mo…
Linear Convergence of Generalized Mirror Descent with Time-Dependent Mirrors
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
The Polyak-Lojasiewicz (PL) inequality is a sufficient condition for establishing linear convergence of gradient descent, even in non-convex settings. While several recent works us…
On Alignment in Deep Linear Neural Networks
Adityanarayanan Radhakrishnan, Eshaan Nichani, Daniel Bernstein +1
We study the properties of alignment, a form of implicit regularization, in linear neural networks under gradient descent. We define alignment for fully connected networks with mul…
Overparameterized Neural Networks Implement Associative Memory
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
Identifying computational mechanisms for memorization and retrieval of data is a long-standing problem at the intersection of machine learning and neuroscience. Our main finding is…