In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness

Liam Collins, Advait Parulekar, Aryan Mokhtari +2

A striking property of transformers is their ability to perform in-context learning (ICL), a machine learning framework in which the learner is presented with a novel context durin…