Building Fast, Evaluating Slow: Pipeline Choices Dominate Autointerpretability Score Variance

Sinie van der Ben, Neele Roch, Anna Hedström +1

Cross-paper comparison of sparse autoencoder (SAE) interpretability often relies on autointerpretability scores. In this evaluation pipeline, a language model (LM) explains each fe…