Progressive Localisation in Localist LLMs

Joachim Diederich

This paper demonstrates that progressive localization, the gradual increase of attention locality from early distributed layers to late localized layers, represents the optimal arc…