Distinct mechanisms underlying in-context learning in transformers

Cole Gibson, Wenping Cui, Gautam Reddy

Modern distributed networks, notably transformers, acquire a remarkable ability (termed `in-context learning') to adapt their computation to input statistics, such that a fixed net…