Inductive Representation Learning on Temporal Graphs

Abstract

Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes as well as capturing temporal patterns. The node embeddings, which are now functions of time, should represent both the static node features and the evolving topological structures. Moreover, node and topological features can be temporal as well, whose patterns the node embeddings should also capture. We propose the temporal graph attention (TGAT) layer to efficiently aggregate temporal-topological neighborhood features as well as to learn the time-feature interactions. For TGAT, we use the self-attention mechanism as building block and develop a novel functional time encoding technique based on the classical Bochner's theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to include the temporal edge features. We evaluate our method with transductive and inductive tasks under temporal settings with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-the-art baselines as well as the previous temporal graph embedding approaches.

References in corpus (12)

• Auto-Encoding Variational Bayes
• Semi-Supervised Classification with Graph Convolutional Networks
• Distributed Representations of Sentences and Documents
• Deep Convolutional Networks on Graph-Structured Data
• Variational Inference with Normalizing Flows
• Fast Graph Representation Learning with PyTorch Geometric
• Variational Graph Auto-Encoders
• Density estimation using Real NVP
• End-To-End Memory Networks
• DynGEM: Deep Embedding Method for Dynamic Graphs
• Representation Learning over Dynamic Graphs
• DyLink2Vec: Effective Feature Representation for Link Prediction in Dynamic Networks

Cited by in corpus (13)

• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks
• Lifelong Learning of Graph Neural Networks for Open-World Node Classification
• Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
• Event2Graph: Event-driven Bipartite Graph for Multivariate Time-series Anomaly Detection
• Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs
• DyFormer: A Scalable Dynamic Graph Transformer with Provable Benefits on Generalization Ability
• Learning to Pool in Graph Neural Networks for Extrapolation
• Deep Structural Point Process for Learning Temporal Interaction Networks
• ConTIG: Continuous Representation Learning on Temporal Interaction Graphs
• Robust Dynamic Network Embedding via Ensembles
• FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding
• TME-BNA: Temporal Motif-Preserving Network Embedding with Bicomponent Neighbor Aggregation