In-Context Convergence of Transformers

Yu Huang, Yuan Cheng, Yingbin Liang

Transformers have recently revolutionized many domains in modern machine learning and one salient discovery is their remarkable in-context learning capability, where models can sol…