Assign and Add: A Mechanistic Study of Compositional Arithmetic

Brady Exoo, Alberto Bietti, John Sous

Large language models are able to compose skills in order to perform complex tasks, many of which might not have been seen during training. The details of how exactly this composit…