Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
SO(3)-RoPE for Spherical Transformers
Christian Libner, Chase van de Geijn, Alexander S. Ecker +1
Spherical data arise in many scientific applications. Often spherical transformers disregard the geometry of the underlying spherical domain, causing distortions and coordinate sin…
cs.LG2025
Do traveling waves make good positional encodings?
Chase van de Geijn, Ayush Paliwal, Timo Lüddecke +1
Transformers rely on positional encoding to compensate for the inherent permutation invariance of self-attention. Traditional approaches use absolute sinusoidal embeddings or learn…
cs.LG2025
Clifford Algebraic Rotor Embeddings : Maybe embeddings should start to CARE
Sameeksha Sriram, Ayush Paliwal, Alexander S. Ecker +1
Rotary Positional Embeddings (RoPE) have demonstrated exceptional performance as a positional encoding method, consistently outperforming their baselines. While recent work has sou…