From Copying to Composition: How Language Models Develop In-Context Learning

Minglai Yang, Xinyu Guo, Renxiong Wang +3

In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual informa…