From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers

Binbin Lin, Wei Chen, Yalun Li +3

Self-attention is a ubiquitous primitive in modern sequence models, yet its operator-level geometry is only partially understood. We view a token sequence as a vector field over th…