Mechanistic Fine-tuning for In-context Learning

Hakaze Cho, Peng Luo, Mariko Kato +2

In-context Learning (ICL) utilizes structured demonstration-query inputs to induce few-shot learning on Language Models (LMs), which are not originally pre-trained on ICL-style dat…