Soft Head Selection for Injecting ICL-Derived Task Embeddings

Jungwon Park, Jimyeong Kim, Changin Choi +1

Large language models (LLMs) are commonly adapted to downstream tasks using parameter-efficient fine-tuning (PEFT) or in-context learning (ICL). Recently, ICL-driven embedding-base…