What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning

Jane Pan, Tianyu Gao, Howard Chen +1

Large language models (LLMs) exploit in-context learning (ICL) to solve tasks with only a few demonstrations, but its mechanisms are not yet well-understood. Some works suggest tha…