Algorithmic Phase Transitions in Language Models: A Mechanistic Case Study of Arithmetic

Alan Sun, Ethan Sun, Warren Shepard

Zero-shot capabilities of large language models make them powerful tools for solving a range of tasks without explicit training. It remains unclear, however, how these models achie…