Understanding Task Vectors in In-Context Learning: Emergence, Functionality, and Limitations

Yuxin Dong, Jiachen Jiang, Zhihui Zhu +1

Task vectors offer a compelling mechanism for accelerating inference in in-context learning (ICL) by distilling task-specific information into a single, reusable representation. De…