2 papers
cs.LG2026
Inherent Trade-offs when Enforcing Correct Data Symmetries in Multi-head Self-attention
Tīkun Ông, Georg Bökman
We give a complete characterization of equivariant multi-head self-attention (MHSA): if an MHSA layer is equivariant to a symmetry group , then can only act by permuting hea…
cs.CV2026
A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of
Tīkun Ông, Georg Bökman
Vision transformers have become a dominant architecture for visual recognition. However, standard models do not explicitly encode the planar symmetries that arise in many vision do…