A Circuit for In-Context Learning

Andrew Gao, William Gao

In-context learning (ICL) lets a pretrained language model perform a task specified by demonstrations in its prompt, with no parameter updates. With arbitrary labels the model must…