Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression

Mingsong Yan, Dongyang Li, Charles Kulick +1

Mechanistic accounts of in-context learning (ICL) have identified iterative algorithms for linear regression and related linear prediction tasks, often using linear or ReLU attenti…