Associative Transformer

Yuwei Sun, Hideya Ochiai, Zhirong Wu +2

Emerging from the pairwise attention in conventional Transformers, there is a growing interest in sparse attention mechanisms that align more closely with localized, contextual lea…