Universal Response and Emergence of Induction in LLMs

Niclas Luick

While induction is considered a key mechanism for in-context learning in LLMs, understanding its precise circuit decomposition beyond toy models remains elusive. Here, we study the…