Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders

Siyu Chen, Heejune Sheen, Xuyuan Xiong +2

We study the challenge of achieving theoretically grounded feature recovery using Sparse Autoencoders (SAEs) for the interpretation of Large Language Models. Existing SAE training…