ZetaGPT: A Reference Implementation of Positional--Encoding--Free State--Space--Attention Language Models

Róisín Luo

Transformer-based language models rely on self-attention, whose computation is permutation-equivariant and therefore lacks an intrinsic mechanism for representing token order. Exis…