Conformal Transformations for Symmetric Power Transformers

Saurabh Kumar, Jacob Buckman, Carles Gelada +1

Transformers with linear attention offer significant computational advantages over softmax-based transformers but often suffer from degraded performance. The symmetric power (sympo…