ExPe: Exact Positional Encodings for Generative Transformer Models with Extrapolating Capabilities

Aleksis Datseris, Sylvia Vassileva, Ivan Koychev +1

This paper introduces a novel approach to position embeddings in transformer models, named "Exact Positional Embeddings" (ExPE). An absolute positional embedding method that can ex…