The Independence Prior of SAEs Fragments Visual Concepts

Tommaso Mencattini, Giorgos Nikolaou, Donato Crisostomi +4

Sparse Autoencoders (SAEs) decompose model activations into sparse combinations of interpretable dictionary atoms. Although SAEs are grounded in the Linear Representation Hypothesi…