MEP: Multiple Kernel Learning Enhancing Relative Positional Encoding Length Extrapolation

Weiguo Gao

When the predicted sequence length exceeds the length seen during training, the transformer's inference accuracy diminishes. Existing relative position encoding methods, such as th…