Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers

Thomas Heap, Tim Lawson, Lucy Farnik +1

Sparse autoencoders (SAEs) are widely used to extract sparse, interpretable latents from transformer activations. We test whether commonly used SAE quality metrics and automatic ex…