Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning

Jeffrey Olmo, Jared Wilson, Max Forsey +3

Sparse Autoencoders (SAEs) are a promising approach for extracting neural network representations by learning a sparse and overcomplete decomposition of the network's internal acti…