Individual Parameters in Weight-Sparse Transformers Appear Interpretable

Arnau Marin-Llobet, Stefan Heimersheim

A central goal of mechanistic interpretability is to understand how neural networks work and what each individual component does. Dominant circuit-finding approaches focus on a spe…