An Analysis of Residual-Stream Geometry Across Transformer Depth

Sunit Bhattacharya, Ravi Shankar Kolli

We propose a transition-centred geometric analysis of transformer residual streams. Relative displacement measures how \emph{far} representations move between consecutive layers, a…