Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders

Charles O'Neill, Alim Gumran, David Klindt

A recent line of work has shown promise in using sparse autoencoders (SAEs) to uncover interpretable features in neural network representations. However, the simple linear-nonlinea…