Feature Starvation as Geometric Instability in Sparse Autoencoders

Faris Chaudhry, Keisuke Yano, Anthea Monod

Sparse autoencoders (SAEs) are used to disentangle the dense, polysemantic internal representations of large language models (LLMs) into interpretable, monosemantic concepts. Howev…