TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer

Abstract

Due to the proficiency of self-attention mechanisms (SAMs) in capturing dependencies in sequence modeling, several existing dynamic graph neural networks (DGNNs) utilize Transformer architectures with various encoding designs to capture sequential evolutions of dynamic graphs. However, the effectiveness and efficiency of these Transformer-based DGNNs vary significantly, highlighting the importance of properly defining the SAM on dynamic graphs and comprehensively encoding temporal and interactive dynamics without extra complex modules. In this work, we propose TIDFormer, a dynamic graph TransFormer that fully exploits Temporal and Interactive Dynamics in an efficient manner. We clarify and verify the interpretability of our proposed SAM, addressing the open problem of its uninterpretable definitions on dynamic graphs in previous works. To model the temporal and interactive dynamics, respectively, we utilize the calendar-based time partitioning information and extract informative interaction embeddings for both bipartite and non-bipartite graphs using merely the sampled first-order neighbors. In addition, we jointly model temporal and interactive features by capturing potential changes in historical interaction patterns through a simple decomposition. We conduct extensive experiments on several dynamic graph datasets to verify the effectiveness and efficiency of TIDFormer. The experimental results demonstrate that TIDFormer excels, outperforming state-of-the-art models across most datasets and experimental settings. Furthermore, TIDFormer exhibits significant efficiency advantages compared to previous Transformer-based methods.

KDD2025

References in corpus (8)

• Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks
• Evolutionary dynamics of higher-order interactions in social networks
• Session-based Social Recommendation via Dynamic Graph Attention Networks
• Dynamic Graph Convolutional Networks
• APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding
• Representation Learning for Dynamic Graphs: A Survey
• Towards Better Dynamic Graph Learning: New Architecture and Unified Library
• Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks