Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts

Anna Mészáros, Szilvia Ujváry, Wieland Brendel +2

LLMs show remarkable emergent abilities, such as inferring concepts from presumably out-of-distribution prompts, known as in-context learning. Though this success is often attribut…