3 citations · 4 across the 8 of their papers we have counts for
9 papers
Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis +1
Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a…
On-Policy Distillation of Language Models for Autonomous Vehicle Motion Planning
Amirhossein Afsharrad, Amirhesam Abedsoltan, Ahmadreza Moradipari +1
Large language models (LLMs) have recently demonstrated strong potential for autonomous vehicle motion planning by reformulating trajectory prediction as a language generation prob…
Task Generalization With AutoRegressive Compositional Structure: Can Learning From Tasks Generalize to Tasks?
Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen +3
Large language models (LLMs) exhibit remarkable task generalization, solving tasks they were never explicitly trained on with only a few demonstrations. This raises a fundamental q…
Fast training of large kernel models with delayed projections
Amirhesam Abedsoltan, Siyuan Ma, Parthe Pandit +1
Classical kernel machines have historically faced significant challenges in scaling to large datasets and model sizes--a key ingredient that has driven the success of neural networ…
Context-Scaling versus Task-Scaling in In-Context Learning
Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1
Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…
On the Nystrom Approximation for Preconditioning in Kernel Machines
Amirhesam Abedsoltan, Parthe Pandit, Luis Rademacher +1
Kernel methods are a popular class of nonlinear predictive models in machine learning. Scalable algorithms for learning kernel models need to be iterative in nature, but convergenc…
On Emergence of Clean-Priority Learning in Early Stopped Neural Networks
Chaoyue Liu, Amirhesam Abedsoltan, Mikhail Belkin
When random label noise is added to a training dataset, the prediction error of a neural network on a label-noise-free test dataset initially improves during early training but eve…
Toward Large Kernel Models
Amirhesam Abedsoltan, Mikhail Belkin, Parthe Pandit
Recent studies indicate that kernel machines can often perform similarly or better than deep neural networks (DNNs) on small datasets. The interest in kernel machines has been addi…
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
Neil Mallinar, James B. Simon, Amirhesam Abedsoltan +3
The practical success of overparameterized neural networks has motivated the recent scientific study of interpolating methods, which perfectly fit their training data. Certain inte…