WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing

Wenbo Zhang, Xiang Ren

When generating text, a Transformer produces representations of past tokens at every layer, but each layer can normally use only representations from the same depth. This restricti…