Fusion Matters: Length-Aware Analysis of Positional-Encoding Fusion in Transformers

Mohamed Amine Hallam, Kuo-Kun Tseng

Transformers require positional encodings to represent sequence order, yet most prior work focuses on designing new positional encodings rather than examining how positional inform…