From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoders

Yifan Luo, Yang Zhan, Jiedong Jiang +4

Sparse autoencoders (SAEs) have proven effective for extracting monosemantic features from large language models (LLMs), yet these features are typically identified in isolation. H…