Behavioral Compositionality of Sparse Autoencoder Representations

Haotian Ma, Ziye Mei, Frederic Sala

Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively. We introduc…