ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection

Qingwen Zeng, Zehao Fu, Shuyu Meng +5

Sparse autoencoders (SAEs) are a central tool in mechanistic interpretability. However, existing SAEs are primarily trained per layer. The modeling subspace is therefore fixed by l…