Task Vectors in In-Context Learning: Emergence, Formation, and Benefit

Liu Yang, Ziqian Lin, Kangwook Lee +2

In-context learning is a remarkable capability of transformers, referring to their ability to adapt to specific tasks based on a short history or context. Previous research has fou…