Global Attention Improves Graph Networks Generalization

Omri Puny, Heli Ben-Hamu, Yaron Lipman

This paper advocates incorporating a Low-Rank Global Attention (LRGA) module, a computation and memory efficient variant of the dot-product attention (Vaswani et al., 2017), to Gra…