Geometric and Behavioral Stratification in Transformer Residual Streams

Nelson Guda

Trained transformer models develop privileged bases: coordinate axes whose statistics differ from the rest of the residual stream. But what kind of direction does such a basis sele…