42 citations · 44 across the 2 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023★ 2 cited
Self-Attention in Colors: Another Take on Encoding Graph Structure in Transformers
Romain Menegaux, Emmanuel Jehanno, Margot Selosse +1
We introduce a novel self-attention mechanism, which we call CSA (Chromatic Self-Attention), which extends the notion of attention scores to attention _filters_, independently modu…
cs.LG2021★ 42 cited
GraphiT: Encoding Graph Structure in Transformers
Grégoire Mialon, Dexiong Chen, Margot Selosse +1
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations l…