72 citations · 98 across the 14 of their papers we have counts for
3 papers · 2 filters
Decomposing The Dark Matter of Sparse Autoencoders
Joshua Engels, Logan Riggs, Max Tegmark
Sparse autoencoders (SAEs) are a promising technique for decomposing language model activations into interpretable linear features. However, current SAEs fall short of completely e…
Investigating Representation Universality: Case Study on Genealogical Representations
David D. Baek, Yuxiao Li, Max Tegmark
Motivated by interpretability and reliability, we investigate whether large language models (LLMs) deploy universal geometric structures to encode discrete, graph-structured knowle…
Efficient Dictionary Learning with Switch Sparse Autoencoders
Anish Mudide, Joshua Engels, Eric J. Michaud +2
Sparse autoencoders (SAEs) are a recent technique for decomposing neural network activations into human-interpretable features. However, in order for SAEs to identify all features…