Discovering Millions of Interpretable Features with Sparse Autoencoders

XinYang He, Wei Wang, Bing Zhao +5

Sparse autoencoders (SAEs) have emerged as a powerful tool for decomposing superposed language model representations into sparse and interpretable features. However, training SAEs…