Toward Manifest Relationality in Transformers via Symmetry Reduction

J. François, L. Ravera

Transformer models contain substantial internal redundancy arising from coordinate-dependent representations and continuous symmetries, in model space and in head space, respective…