Tokenized SAEs: Disentangling SAE Reconstructions

Thomas Dooms, Daniel Wilhelm

Sparse auto-encoders (SAEs) have become a prevalent tool for interpreting language models' inner workings. However, it is unknown how tightly SAE features correspond to computation…