Generalized Attention Mechanism and Relative Position for Transformer

R. V. R. Pandya

In this paper, we propose generalized attention mechanism (GAM) by first suggesting a new interpretation for self-attention mechanism of Vaswani et al. . Following the interpretati…