Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight

Haolin Yang, Hakaze Cho, Kaize Ding +1

Large Language Models (LLMs) can perform new tasks from in-context demonstrations, a phenomenon known as in-context learning (ICL). Recent work suggests that these demonstrations a…