Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention

Li Hengyu

The pre-softmax score of an attention head is a bilinear form $score(i,j)=x_i^{T}M{x}_j$ in a learned operator $M=W_q^T{W}_k$. Because M is generally non-symmetric, hence non-norm…