Navigating the Concept Space of Language Models

Wilson E. Marcílio-Jr, Danilo M. Eler

Sparse autoencoders (SAEs) trained on large language model activations output thousands of features that enable mapping to human-interpretable concepts. The current practice for an…