Scaling and evaluating sparse autoencoders

Leo Gao, Tom Dupré la Tour, Henk Tillman +6

Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck lay…