Sparsification and Reconstruction from the Perspective of Representation Geometry

Wenjie Sun, Bingzhe Wu, Zhile Yang +1

Sparse Autoencoders (SAEs) have emerged as a predominant tool in mechanistic interpretability, aiming to identify interpretable monosemantic features. However, how does sparse enco…