Spectral Filters, Dark Signals, and Attention Sinks

Nicola Cancedda

Projecting intermediate representations onto the vocabulary is an increasingly popular interpretation tool for transformer-based LLMs, also known as the logit lens. We propose a qu…