69 citations · 217 across the 26 of their papers we have counts for
27 papers
Exact Dynamics and Finite-Sample Trajectory Recovery of Linear Recursive Feature Machines
Andrew Cheng, Bobak T. Kiani, Yue M. Lu +1
Recursive feature machines (RFMs) learn representations of data by alternating between fitting a predictor to a dataset and updating features of that predictor using the average gr…
The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks
Taehun Cha, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1
Understanding how deep neural networks learn representations remains a central challenge in machine learning theory. In this work, we propose a feature-centric framework for analyz…
Contextual Linear Activation Steering of Language Models
Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1
Linear activation steering is a powerful approach for eliciting the capabilities of large language models and specializing their behavior using limited labeled data. While effectiv…
Efficient and accurate steering of Large Language Models through attention-guided feature learning
Parmida Davarmanesh, Ashia Wilson, Adityanarayanan Radhakrishnan
Steering, or direct manipulation of internal activations to guide LLM responses toward specific semantic concepts, is emerging as a promising avenue for both understanding how sema…
xRFM: Accurate, scalable, and interpretable feature learning models for tabular data
Daniel Beaglehole, David Holzmüller, Adityanarayanan Radhakrishnan +1
Inference from tabular data, collections of continuous and categorical variables organized into matrices, is a foundation for modern technology and science. Yet, in contrast to the…
Toward universal steering and monitoring of AI models
Daniel Beaglehole, Adityanarayanan Radhakrishnan, Enric Boix-Adserà +1
Modern AI models contain much of human knowledge, yet understanding of their internal representation of this knowledge remains elusive. Characterizing the structure and properties…
Context-Scaling versus Task-Scaling in In-Context Learning
Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1
Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…
Emergence in non-neural models: grokking modular arithmetic via average gradient outer product
Neil Mallinar, Daniel Beaglehole, Libin Zhu +3
Neural networks trained to solve modular arithmetic tasks exhibit grokking, a phenomenon where the test accuracy starts improving long after the model achieves 100% training accura…
Linear Recursive Feature Machines provably recover low-rank matrices
Adityanarayanan Radhakrishnan, Mikhail Belkin, Dmitriy Drusvyatskiy
A fundamental problem in machine learning is to understand how neural networks make accurate predictions, while seemingly bypassing the curse of dimensionality. A possible explanat…
Mechanism of feature learning in convolutional neural networks
Daniel Beaglehole, Adityanarayanan Radhakrishnan, Parthe Pandit +1
Understanding the mechanism of how convolutional neural networks learn features from image data is a fundamental problem in machine learning and computer vision. In this work, we i…
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning
Libin Zhu, Chaoyue Liu, Adityanarayanan Radhakrishnan +1
In this paper, we first present an explanation regarding the common occurrence of spikes in the training loss when neural networks are trained with stochastic gradient descent (SGD…
Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features
Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit +1
In recent years neural networks have achieved impressive results on many technological and scientific tasks. Yet, the mechanism through which these models automatically select feat…
Transfer Learning with Kernel Methods
Adityanarayanan Radhakrishnan, Max Ruiz Luyten, Neha Prasad +1
Transfer learning refers to the process of adapting a model trained on a source task to a target task. While kernel methods are conceptually and computationally simple machine lear…
Quadratic models for understanding catapult dynamics of neural networks
Libin Zhu, Chaoyue Liu, Adityanarayanan Radhakrishnan +1
While neural networks can be approximated by linear models as their width increases, certain properties of wide neural networks cannot be captured by linear models. In this work we…
Wide and Deep Neural Networks Achieve Optimality for Classification
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
While neural networks are used for classification tasks across domains, a long-standing open problem in machine learning is determining whether neural networks trained using standa…
Local Quadratic Convergence of Stochastic Gradient Descent with Adaptive Step Size
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
Establishing a fast rate of convergence for optimization methods is crucial to their applicability in practice. With the increasing popularity of deep learning over the past decade…
Simple, Fast, and Flexible Framework for Matrix Completion with Infinite Width Neural Networks
Adityanarayanan Radhakrishnan, George Stefanakis, Mikhail Belkin +1
Matrix completion problems arise in many applications including recommendation systems, computer vision, and genomics. Increasingly larger neural networks have been successful in m…
A Mechanism for Producing Aligned Latent Spaces with Autoencoders
Saachi Jain, Adityanarayanan Radhakrishnan, Caroline Uhler
Aligned latent spaces, where meaningful semantic shifts in the input space correspond to a translation in the embedding space, play an important role in the success of downstream t…
Increasing Depth Leads to U-Shaped Test Risk in Over-parameterized Convolutional Networks
Eshaan Nichani, Adityanarayanan Radhakrishnan, Caroline Uhler
Recent works have demonstrated that increasing model capacity through width in over-parameterized neural networks leads to a decrease in test risk. For neural networks, however, mo…
Linear Convergence of Generalized Mirror Descent with Time-Dependent Mirrors
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
The Polyak-Lojasiewicz (PL) inequality is a sufficient condition for establishing linear convergence of gradient descent, even in non-convex settings. While several recent works us…
Causal Network Models of SARS-CoV-2 Expression and Aging to Identify Candidates for Drug Repurposing
Anastasiya Belyaeva, Louis Cammarata, Adityanarayanan Radhakrishnan +4
Given the severity of the SARS-CoV-2 pandemic, a major challenge is to rapidly repurpose existing approved drugs for clinical interventions. While a number of data-driven and exper…
On Alignment in Deep Linear Neural Networks
Adityanarayanan Radhakrishnan, Eshaan Nichani, Daniel Bernstein +1
We study the properties of alignment, a form of implicit regularization, in linear neural networks under gradient descent. We define alignment for fully connected networks with mul…
Overparameterized Neural Networks Implement Associative Memory
Adityanarayanan Radhakrishnan, Mikhail Belkin, Caroline Uhler
Identifying computational mechanisms for memorization and retrieval of data is a long-standing problem at the intersection of machine learning and neuroscience. Our main finding is…
Memorization in Overparameterized Autoencoders
Adityanarayanan Radhakrishnan, Karren Yang, Mikhail Belkin +1
The ability of deep neural networks to generalize well in the overparameterized regime has become a subject of significant research interest. We show that overparameterized autoenc…