Showing 2026Show all
3 papers · 1 filter
stat.ML2026
Learning to Recall with Transformers Beyond Orthogonal Embeddings
Nuri Mert Vural, Alberto Bietti, Mahdi Soltanolkotabi +1
Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this ca…
cs.LG2026
Training Dynamics of Softmax Self-Attention: Fast Global Convergence via Preconditioning
Gautam Goel, Mahdi Soltanolkotabi, Peter Bartlett
We study the training dynamics of gradient descent in a multilayer softmax self-attention network trained to perform linear regression and show that in the infinite-data limit this…
cs.LG2026
CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
Zarif Ikram, Arad Firouzkouhi, Stephen Tu +2
A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corr…