In-Context Learning Creates Task Vectors

Roee Hendel, Mor Geva, Amir Globerson

In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In partic…