Readout Scaling Shapes Sparse Feature Recovery

Brian Xiao, Brain Xiao, Sarah Wilson

Readout scaling changes what a sparse autoencoder (SAE) learns, but the benefit of removing it depends on the data. We use readout-canonicalized decision-space (RC-DS) coordinates…