The Communication Map of a Transformer

Richard Zhe Wang

The components of a transformer communicate by writing to and reading from a shared residual stream, and mechanistic interpretability has mapped these connections by hand, one circ…