Robustness of Similarity-based Positional Encoding Under Rotations: Theoretical Analysis and Experimental Validation

Andrea Santomauro, Luigi Portinale, Giorgio Leonardi

Positional encoding is a fundamental component of Transformer architectures, as it injects information about the spatial or sequential arrangement of inputs. Among recent alternati…