Continuum Transformers Perform In-Context Learning by Operator Gradient Descent

Abhiti Mishra, Yash Patel, Ambuj Tewari

Transformers robustly exhibit the ability to perform in-context learning, whereby their predictive accuracy on a task can increase not by parameter updates but merely with the plac…