Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens

Matteo He, William F. Shen, Xinchi Qiu +1

A language model's prediction of its next token develops across layers, and lens methods track this process by decoding intermediate hidden states into tokens. But a lens reading r…