Parallel Structures in Pre-training Data Yield In-Context Learning

Yanda Chen, Chen Zhao, Zhou Yu +2

Pre-trained language models (LMs) are capable of in-context learning (ICL): they can adapt to a task with only a few examples given in the prompt without any parameter update. Howe…