Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

Yang Li, Si Si, Gang Li +2

Attentional mechanisms are order-invariant. Positional encoding is a crucial component to allow attention-based deep model architectures such as Transformer to address sequences or…