Query Lens: Interpreting Sparse Key-Value Features with Indirect Effects

Hwiyeong Lee, Ingyu Bang, Uiji Hwang +2

While sparse autoencoders provide features more interpretable than individual neurons, reliably characterizing them remains challenging. We propose Query Lens, which extends Logit…