Causal Optimal-Transport Attention: Positional Priors and Length Extrapolation

Ming Lian

Causal masking obstructs the column balancing used by optimal-transport (OT) attention. We study a prefix-consistent unbalanced operator and characterize the positional prior it in…