Understanding In-context Learning of Addition via Activation Subspaces

Xinyan Hu, Kayo Yin, Michael I. Jordan +2

To perform few-shot learning, language models extract signals from a few input-label pairs, aggregate these into a learned prediction rule, and apply this rule to new inputs. How i…