Modularity in Transformers: Investigating Neuron Separability & Specialization

Nicholas Pochinkov, Thomas Jones, Mohammed Rashidur Rahman

Transformer models are increasingly prevalent in various applications, yet our understanding of their internal workings remains limited. This paper investigates the modularity and…