Symmetry Reveals Layerwise Dynamics: How Transformers Perform In-Context Classification

Patrick Lutz, Themistoklis Haris, Arjun Chandra +2

Transformers can perform in-context classification from a few labeled examples, yet the inference-time algorithm remains opaque. We study multi-class linear classification in the h…