Towards Mechanistic Interpretability of Graph Transformers via Attention Graphs

Batu El, Deepro Choudhury, Pietro Liò +1

We introduce Attention Graphs, a new tool for mechanistic interpretability of Graph Neural Networks (GNNs) and Graph Transformers based on the mathematical equivalence between mess…