4 papers
ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection
Qingwen Zeng, Zehao Fu, Shuyu Meng +5
Sparse autoencoders (SAEs) are a central tool in mechanistic interpretability. However, existing SAEs are primarily trained per layer. The modeling subspace is therefore fixed by l…
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
Xinlei Yu, Zhangquan Chen, Yongbo He +36
Latent space is rapidly emerging as a native substrate for language-based models. While modern systems are still commonly understood through explicit token-level generation, an inc…
Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models
Jiayi Zhang, Shu Yang, Junchao Wu +2
Fine-tuning Large Language Models on a political topic will significantly manipulate their political stance on various issues and unintentionally affect their stance on unrelated t…
Understanding How Value Neurons Shape the Generation of Specified Values in LLMs
Yi Su, Jiayi Zhang, Shu Yang +3
Rapid integration of large language models (LLMs) into societal applications has intensified concerns about their alignment with universal ethical principles, as their internal val…