Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability

Anton Korznikov, Alexey Dontsov, Elena Tutubalina

Mechanistic interpretability aims to decompose neural network computations into human-understandable features, yet progress is limited by the absence of ground-truth units to evalu…