Transformer Layers as Painters

Qi Sun, Marc Pickett, Aakash Kumar Nain +1

Despite their nearly universal adoption for large language models, the internal workings of transformers are not well understood. We aim to better understand the impact of removing…