Higher-Order Modular Attention: Fusing Pairwise and Triadic Interactions for Protein Sequences

Shirin Amiraslani, Xin Gao

Transformer self-attention computes pairwise token interactions, yet protein sequence to phenotype relationships often involve cooperative dependencies among three or more residues…