Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations

Shruti Joshi, Andrea Dittadi, Sébastien Lachapelle +1

Unsupervised approaches to large language model (LLM) interpretability, such as sparse autoencoders (SAEs), offer a way to decode LLM activations into interpretable and, ideally, c…